-- Logs begin at Mon 2022-08-01 15:36:06 UTC, end at Mon 2022-08-01 16:58:44 UTC. -- Aug 01 16:24:08.442023 dv-cinder-850321-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 16:24:08.449750 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 16:24:08.451796 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 16:24:08 2022] *** Aug 01 16:24:08.451796 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: nodename: dv-cinder-850321-2-iscsi Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: machine: x86_64 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: clock source: unix Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: pcre jit disabled Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: detected number of CPU cores: 4 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: current working directory: / Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: detected binary path: /usr/bin/uwsgi-core Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: your processes number limit is 47680 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: your memory page size is 4096 bytes Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: detected max file descriptor number: 2048 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: lock engine: pthread robust mutexes Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: thunder lock: enabled Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 16:24:08.455815 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 16:24:08.493184 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: Python main interpreter initialized at 0x556536c719c0 Aug 01 16:24:08.493184 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: python threads support enabled Aug 01 16:24:08.493184 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: your server socket listen backlog is limited to 100 connections Aug 01 16:24:08.493184 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: your mercy for graceful operations on workers is 90 seconds Aug 01 16:24:08.493618 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 16:24:08.493618 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: *** Operational MODE: preforking *** Aug 01 16:24:08.493904 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: *** uWSGI is running in multiple interpreter mode *** Aug 01 16:24:08.493904 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: spawned uWSGI master process (pid: 80335) Aug 01 16:24:08.494093 dv-cinder-850321-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 16:24:08.499788 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: spawned uWSGI worker 1 (pid: 80348, cores: 1) Aug 01 16:24:08.499788 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: spawned uWSGI worker 2 (pid: 80349, cores: 1) Aug 01 16:24:08.499788 dv-cinder-850321-2-iscsi devstack@keystone.service[80335]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 16:24:14.013568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:24:14.013568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: from cryptography.utils import int_from_bytes Aug 01 16:24:14.014333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:24:14.014333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: from cryptography.utils import int_from_bytes Aug 01 16:24:14.016079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:24:14.016079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: from cryptography.utils import int_from_bytes Aug 01 16:24:14.016526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:24:14.016526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: from cryptography.utils import int_from_bytes Aug 01 16:24:14.065377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 16:24:14.065595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 16:24:14.065798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] command line args: [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 16:24:14.066026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 16:24:14.066222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ================================================================================ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 16:24:14.066535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] admin_token = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.066778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config_dir = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.066963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.067142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] config_source = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.067222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 16:24:14.067361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] debug = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.067461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 16:24:14.067529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.067870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] command line args: [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 16:24:14.067870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 16:24:14.067987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ================================================================================ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 16:24:14.068213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] admin_token = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.068695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config_dir = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.069080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.069355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.069596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] config_source = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.069712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.069797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] debug = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.069965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.070142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.070223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.070351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.070507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.070643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.070888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_config_append = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_config_append = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_dir = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.071905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_dir = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.072969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.073979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.074989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.075124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.075203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.075478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.075715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.075911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.075994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] publish_errors = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] publish_errors = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.076922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] standard_threads = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.077921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_journal = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] standard_threads = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_json = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_stderr = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] use_syslog = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.078959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.079213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.079340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_journal = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.079475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.079640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_json = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.079640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_stderr = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.079977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.080112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] use_syslog = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.080239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.080374 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:24:14.080502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.080629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.080773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.080903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.081843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.082110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.082399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.082547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.082675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.082814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.082949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.083970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.084966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.085962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.086891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.087138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.087267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.087393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.087519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.087646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.087874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.088909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.089805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.090907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.091940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.092976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.093102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.093228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.093355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.093490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.093618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.095128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.095299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.095489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.095678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.096010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.096173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.096360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.096554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.096743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.096993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.097194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.097356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.097545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.097775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.097966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.098154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.098415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.098721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.098911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.099098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.099098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.099468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.099468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.099912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.099912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.100238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.100400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.100582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.100716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.100855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.100855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.101947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.102880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.103996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.104963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.105215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.105215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.105457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.105585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.105799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.105932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.106989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.107124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.107253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.107458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.107589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.107718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.107867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.108966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.109896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.110865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.external = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.111980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.112109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.112238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.external = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.112378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.112659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.112806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.password = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.112949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.113201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] auth.token = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.113367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.113504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.113635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.password = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.113771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.113920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] auth.token = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.114906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.115872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.116917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.117983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.118914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.119995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.120914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.121974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.122916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.123968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.124880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.125934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.126064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.126202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.126331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.126460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.126917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.127900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.128966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.129102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.129238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.129392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.129668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.129816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.129948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.130884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131423 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.131965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.132893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.133946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.134876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.135994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.136976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.137889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.138941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.139873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.140951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.141872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.142911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.143968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.144108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.144281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.144515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.144709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.144890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.145983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.146897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.147888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.148951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.149878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.150197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.150342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.150474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.150604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.150733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.150861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.151958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.152997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.153917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.154828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.155942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.156870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.157917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.158968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.159929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.160924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.161977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.162906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.163855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.164889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.165976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.166904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.167948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.168914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.169903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.170041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80348) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 16:24:14.170178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:24:14.170178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80349) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 16:24:14.181730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.181963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.182126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.182267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.182514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.182660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.190765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.191140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.221035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.221345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.221497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.221637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.275495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.276427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.326875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.327236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.327392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.327579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.393451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.393778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.393930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.394079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.394332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.394485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.394627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.394779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.394919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.395058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.395281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.395281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.417954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.418253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.418409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.418577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.418721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.418861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.419957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.420120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.430606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.430855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.443916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.444251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.444405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.444589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.444734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.444875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80348) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.445866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80349) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:24:14.462412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.462688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:24:14.462841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.462841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:24:14.463113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.463113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:24:14.463392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.463531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.463670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:24:14.463836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.463992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.463992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.464272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.464419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.464559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.464698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.473838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.474084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.474378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.474582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.474735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.474893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.475039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.475195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.481667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.481993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.486464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.486691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.486841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.486987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.487129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.487270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.487428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.487583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.487735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.487910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.488927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.489077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.489229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.489371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.489511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.489511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.512691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.513687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.514162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.514436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.515556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.515972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.519015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.519345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.519628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.521296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.521647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.521995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.526242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.526469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.528935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.529291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.530602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.530889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.531176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.531452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.531721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.532025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.532288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.532559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.534138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.534509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.534935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.535316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.535697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.536081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.536422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.536775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.549017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.549301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.553279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.553409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.553683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.556413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.556706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.557944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.561982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.562108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.562108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.564257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.566184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.567636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.567958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.568247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.568537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.568831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.569119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.570358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.573644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.574013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.574381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.574750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.575111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.575471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.585094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.585373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.585631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.585894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.591999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.592362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.592704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.592884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.593107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.594306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.594574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.597422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.597658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.597980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.598257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.598536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.600700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.602339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.602673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.605593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.605950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.606326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.606682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.607041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.609907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.610186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.610421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.610720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.610991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.611270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.611550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.619224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.619582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.619956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.620311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.620662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.621014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.621375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.627321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.630482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.633405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.636005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.637621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.637897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.638118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.638266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.641281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.644425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.645132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.646714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.647312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.649061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.649422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.649768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.649906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.650036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.650163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.650404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.656608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.658277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.658458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.658598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.658905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.661595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.661956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.662310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.662653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.665888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80349) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.666163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.666437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.666708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.671887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.671887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.671887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.671887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.671887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.671887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.672696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.672696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.672696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.672696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80349) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.677653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.677653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.677653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.679723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80348) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:24:14.680102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.680453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.680792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.681165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.681544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.681891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.682238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.682702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.683077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.683402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.683738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.684059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.684294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80348) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:24:14.702706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556536c719c0 pid: 80349 (default app) Aug 01 16:24:14.706241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28558f13-974d-47e8-8176-817a37648627 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:14.706529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28558f13-974d-47e8-8176-817a37648627 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:14.706799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28558f13-974d-47e8-8176-817a37648627 None None] PATH_INFO: `/v3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:14.712975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Mon Aug 1 16:24:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:14.718442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556536c719c0 pid: 80348 (default app) Aug 01 16:24:22.445447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:22.446056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:22.446056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:22.447311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 2/2] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:22.455202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:22.455650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:22.455741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:22.525959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: INFO dbcounter [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] Registered counter for database keystone Aug 01 16:24:22.545813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG oslo_db.sqlalchemy.engines [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80348) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 16:24:22.553777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writer thread running {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 16:24:22.739680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG passlib.handlers.bcrypt [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80348) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 16:24:22.740056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG passlib.handlers.bcrypt [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80348) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 16:24:22.796283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:22.797912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:22.938386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1/3] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 16:24:23.077866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:23.148208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: INFO dbcounter [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] Registered counter for database keystone Aug 01 16:24:23.170409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG oslo_db.sqlalchemy.engines [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80349) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 16:24:23.178459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writer thread running {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 16:24:23.299286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be659ed0-91b6-4d5b-b3d9-e1ea7106340a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:23.304616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:23.305268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:23.305549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:23.305821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:23.314769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:23.367992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING keystone.common.rbac_enforcer.enforcer [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 16:24:23.368199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e729e2-8c3f-4802-aac8-62e6836cb9d8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:23.379690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 3/4] 10.222.0.45 () {58 vars in 1324 bytes} [Mon Aug 1 16:24:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:24:23.386007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:23.401496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d05a72-2343-43f6-8d8f-41f996f004d7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:23.407310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:23.408523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:23.408804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:23.409075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] PATH_INFO: `/v3/projects/0c8a8aa32f9a4584972956058f337380` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:23.411023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] RBAC: Authorizing `identity:get_project(project_id=0c8a8aa32f9a4584972956058f337380)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:23.462597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 16:24:23.463291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235b0fe-503f-485b-9ec1-100faea82649 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:23.467693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 2/5] 10.222.0.45 () {56 vars in 1207 bytes} [Mon Aug 1 16:24:23 2022] GET /identity/v3/projects/0c8a8aa32f9a4584972956058f337380 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:25.461450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fca5-e363-421f-a05c-91088bc7ee60 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:25.462010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fca5-e363-421f-a05c-91088bc7ee60 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:25.462010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fca5-e363-421f-a05c-91088bc7ee60 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:25.466180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 4/6] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:25.471887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:25.472475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:25.472475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:25.494135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:25.495683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:25.504621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 3/7] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 16:24:25.526907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0fca5-e363-421f-a05c-91088bc7ee60 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:25.540745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0fca5-e363-421f-a05c-91088bc7ee60 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:25.545803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5eda-8d77-41ad-bff4-8c6caf8a987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:25.546426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5eda-8d77-41ad-bff4-8c6caf8a987f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:25.546701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5eda-8d77-41ad-bff4-8c6caf8a987f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:25.546969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5eda-8d77-41ad-bff4-8c6caf8a987f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:25.555749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5eda-8d77-41ad-bff4-8c6caf8a987f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:25.557587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5eda-8d77-41ad-bff4-8c6caf8a987f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:25.566843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 5/8] 10.222.0.45 () {58 vars in 1324 bytes} [Mon Aug 1 16:24:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:24:25.572768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:25.582891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b1dc-3dd0-4027-96c6-de5edf8897f7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:25.588811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc44ab9-0ca2-4a0d-a70c-9927a116e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:25.589444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc44ab9-0ca2-4a0d-a70c-9927a116e4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:25.589748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc44ab9-0ca2-4a0d-a70c-9927a116e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:25.590019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc44ab9-0ca2-4a0d-a70c-9927a116e4ea admin admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:25.591579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc44ab9-0ca2-4a0d-a70c-9927a116e4ea admin admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:25.593504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc44ab9-0ca2-4a0d-a70c-9927a116e4ea admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:25.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 4/9] 10.222.0.45 () {56 vars in 1198 bytes} [Mon Aug 1 16:24:25 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:27.714248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debddef8-4338-4c7b-81af-f63427897dc2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:27.714806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debddef8-4338-4c7b-81af-f63427897dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:27.714806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debddef8-4338-4c7b-81af-f63427897dc2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:27.716188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 6/10] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:27.721471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:27.721904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:27.721997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:27.729875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837247b-6a1a-433f-8d52-a5a23518ded1 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:27.730158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837247b-6a1a-433f-8d52-a5a23518ded1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:27.730434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837247b-6a1a-433f-8d52-a5a23518ded1 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:27.731799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 7/11] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:24:27.736876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25412-5370-495f-8a68-11261df0deac None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:27.737151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25412-5370-495f-8a68-11261df0deac None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:27.737418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25412-5370-495f-8a68-11261df0deac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:27.743098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:27.744348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:27.791691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG passlib.handlers.bcrypt [None req-31a25412-5370-495f-8a68-11261df0deac None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80349) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 16:24:27.792195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG passlib.handlers.bcrypt [None req-31a25412-5370-495f-8a68-11261df0deac None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80349) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 16:24:27.836056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 5/12] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:27.842035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-31a25412-5370-495f-8a68-11261df0deac None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:27.843665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-31a25412-5370-495f-8a68-11261df0deac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:27.858267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:27.887731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:27.956429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 8/13] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:27.973656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a5e4-22cb-4930-881c-4872ad5d8263 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:27.980632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25412-5370-495f-8a68-11261df0deac None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.010164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.010823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.011101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.011375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.082182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25412-5370-495f-8a68-11261df0deac None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.084372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.084727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:28.085864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.112072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:28.121042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.122186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.122455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.122724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.135154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:28.143690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:28.145065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:28.145924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 9/14] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:28.152942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.189690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 6/15] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:28.195816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.263169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faddd1b-e792-479e-bf0b-38798d309426 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.300461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.301085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.301361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.301638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.302441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:28.302756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.304289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.310416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335e104-7a49-49dd-b37b-b59a2ea6e8fe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.317783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 10/16] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:28.323678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.336570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:28.349720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.350387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.350667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.350942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.357143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:28.363277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.364844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:24:28.384656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:28.386002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:28.387009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:24:28.387009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:24:28.387009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:24:28.387009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:24:28.387009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 7/17] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:24:28.393198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.439589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093fbd-7e65-4d07-b0ee-7c4ddab0d066 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.473828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.474504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.474781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.475055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.476776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.478293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.479990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 11/18] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:28 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:28.485884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.590624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:28.598147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e656f-9584-459b-9fce-94f291d7eca1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.607256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38b1d4-ed55-4839-aab5-dd487fd0ed42 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.633951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.634603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.634884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.635155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.636366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.637849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.639451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 12/19] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:28.643743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.644557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.644844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.645110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.645562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:28.645803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:28.646099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.647489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7890b-2ce1-449e-bce9-46d0d57e3fd7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.665139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 8/20] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:28.760747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc81cba-c46a-4870-b4d4-2b6ecc68dfcf None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:28.801488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:28.802256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:28.802526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:28.802774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:28.803582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.domain.id=default, user.id=cc7787b19b7142fdb85a77e20df6ff3d) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:28.803924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:28.805406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e03a05-b1bb-4367-b609-f10ab02133a1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:28.815142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 13/21] 10.222.0.45 () {56 vars in 1322 bytes} [Mon Aug 1 16:24:28 2022] GET /identity/v3/role_assignments?user.id=cc7787b19b7142fdb85a77e20df6ff3d&scope.domain.id=default&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:31.307844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cae74e-d101-4438-aa6b-d88396260a0e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.308852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cae74e-d101-4438-aa6b-d88396260a0e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.309335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cae74e-d101-4438-aa6b-d88396260a0e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.310915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 9/22] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:31.316428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e21c-e699-42db-a61d-08935c65a4dd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.317360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e21c-e699-42db-a61d-08935c65a4dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.317833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e21c-e699-42db-a61d-08935c65a4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.349033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2678e21c-e699-42db-a61d-08935c65a4dd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:31.350637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2678e21c-e699-42db-a61d-08935c65a4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:31.379260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:31.436640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 14/23] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:31.473969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cae74e-d101-4438-aa6b-d88396260a0e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:31.497972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6278-a1d3-4367-af9a-b9b9d7c23e80 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.498277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6278-a1d3-4367-af9a-b9b9d7c23e80 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.498515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6278-a1d3-4367-af9a-b9b9d7c23e80 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.499933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 15/24] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:24:31.509772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbc3d5-6d0e-41b5-a875-d552263d2413 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.510116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbc3d5-6d0e-41b5-a875-d552263d2413 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.510401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbc3d5-6d0e-41b5-a875-d552263d2413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.533954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9fbbc3d5-6d0e-41b5-a875-d552263d2413 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:31.535541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9fbbc3d5-6d0e-41b5-a875-d552263d2413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:31.544601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:31.653839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 16/25] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:31.655463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53b34-891a-4505-92a1-d9b9956ada1b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.655971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53b34-891a-4505-92a1-d9b9956ada1b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.656463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53b34-891a-4505-92a1-d9b9956ada1b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.657925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 17/26] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:31.665477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cae74e-d101-4438-aa6b-d88396260a0e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:31.665872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8900f7-9ba0-42a2-912d-45cd30b14299 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.666254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8900f7-9ba0-42a2-912d-45cd30b14299 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.666633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8900f7-9ba0-42a2-912d-45cd30b14299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.696077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8a8900f7-9ba0-42a2-912d-45cd30b14299 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:31.699344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8a8900f7-9ba0-42a2-912d-45cd30b14299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:31.744871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:31.745822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.746246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.746662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.764109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:31.765830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:24:31.782053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:31.783079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:31.784153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:24:31.784153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:24:31.784153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:24:31.784153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:24:31.786390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 10/27] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:31.787388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:31.811785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 18/28] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:31.813613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636041-22ab-4236-94f6-e564b34f960e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.813901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636041-22ab-4236-94f6-e564b34f960e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.814176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636041-22ab-4236-94f6-e564b34f960e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.815808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 19/29] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:31.820456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a636041-22ab-4236-94f6-e564b34f960e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:31.844567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:31.899292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:31.923714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e29cf5-d884-4899-b33b-49ad84fab712 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:31.947915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a636041-22ab-4236-94f6-e564b34f960e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:31.965626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399d25e-e371-40a1-9826-306b5254b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:31.966621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399d25e-e371-40a1-9826-306b5254b6f1 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:31.967074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399d25e-e371-40a1-9826-306b5254b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:31.967513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399d25e-e371-40a1-9826-306b5254b6f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:31.968558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399d25e-e371-40a1-9826-306b5254b6f1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:31.970206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399d25e-e371-40a1-9826-306b5254b6f1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:32.107239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 11/30] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 16:24:31 2022] POST /identity/v3/roles => generated 219 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:32.109032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.109505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.109981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.119208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:32.119634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.120008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.120348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.121138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:32.121530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:32.123026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:32.132741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:32.135618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:32.164229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 20/31] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:32.165192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:32.221772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 12/32] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:32.223114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:32.316216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:32.332293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:32.350014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877044c-2cc3-4d39-9126-910f9e7ee821 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:32.371929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732fc0c-01c6-4eb8-ae71-769f34ff0239 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:32.391174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:32.391974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.392262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.392537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.408558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:32.419936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:32.420845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 21/33] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:32.421587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:32.431529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:32.432584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.433083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.433549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.434475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:32.436368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:32.488367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 13/34] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:24:32 2022] POST /identity/v3/projects => generated 301 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:32.490211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:32.643001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feafb51-590a-4793-bf3e-6bc966719de4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:32.686504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24f151-e0c4-4015-858b-ca11abfaaa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:32.686936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24f151-e0c4-4015-858b-ca11abfaaa88 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.687223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24f151-e0c4-4015-858b-ca11abfaaa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.687499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24f151-e0c4-4015-858b-ca11abfaaa88 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.689051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24f151-e0c4-4015-858b-ca11abfaaa88 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:32.692264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24f151-e0c4-4015-858b-ca11abfaaa88 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:32.697020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc6df1-0dbf-45dc-b096-c29bbd831075 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:32.700065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:32.742225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:32.752766 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:32.754268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.754846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.755172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.755977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:32.756286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:32.757704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce066c-e894-4eb8-9e7e-c769f925a2c5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:32.772273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 14/35] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:32.773549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc78d3-0aa3-42b4-94f4-3216266be4d0 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.773838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc78d3-0aa3-42b4-94f4-3216266be4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.774118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc78d3-0aa3-42b4-94f4-3216266be4d0 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.775384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 15/36] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:32.776382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6928f-b791-496b-85ca-6c7368f7e443 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.776662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6928f-b791-496b-85ca-6c7368f7e443 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.776936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6928f-b791-496b-85ca-6c7368f7e443 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.778132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 16/37] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:32.778883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6928f-b791-496b-85ca-6c7368f7e443 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:32.830394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 22/38] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 16:24:32 2022] POST /identity/v3/roles => generated 225 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:32.832383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585df58-492f-44fb-a9b1-af95ccfe5c35 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.832767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585df58-492f-44fb-a9b1-af95ccfe5c35 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.833049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585df58-492f-44fb-a9b1-af95ccfe5c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.854645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6585df58-492f-44fb-a9b1-af95ccfe5c35 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:32.856352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6585df58-492f-44fb-a9b1-af95ccfe5c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:32.919187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6928f-b791-496b-85ca-6c7368f7e443 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:32.938057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 23/39] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:32.939381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.939672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.939982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.960353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:32.962047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:32.980499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:32.981995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:32.982437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:32.982854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:32.984306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:32.986136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:32.988417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 17/40] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:32 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:32.989842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:33.056397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 24/41] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:33.057274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:33.066009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:33.112203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531181c-2e24-469d-a3db-8eaf252a03f4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:33.124608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:33.147306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:33.147974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:33.148256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:33.148526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:33.149259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:33.150672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:33.166700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 18/42] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 16:24:32 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:33.167571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:33.173840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e2c79-3f52-4a50-8115-8e4c81149cf8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:33.218011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:33.218995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:33.219314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:33.219556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:33.221041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:33.223168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:33.225224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 25/43] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:33.231213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:33.292773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd213b-e0fd-4ddb-9cfa-1eb68151f84c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:33.314158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:33.338595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:33.339251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:33.339518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:33.339868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:33.341076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:33.342569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:33.344238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 19/44] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:33.350470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:33.351009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b72754-517c-44fa-a2e9-b750bfd9aa58 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:33.386156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:33.397857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc4fd4-c638-426a-a5b2-66659f11ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:33.398554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4fd4-c638-426a-a5b2-66659f11ea05 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:33.398830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4fd4-c638-426a-a5b2-66659f11ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:33.399101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4fd4-c638-426a-a5b2-66659f11ea05 None admin] PATH_INFO: `/v3/domains/default/users/cc7787b19b7142fdb85a77e20df6ff3d/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:33.401134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc4fd4-c638-426a-a5b2-66659f11ea05 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc7787b19b7142fdb85a77e20df6ff3d, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:33.402610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc4fd4-c638-426a-a5b2-66659f11ea05 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:33.420115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 26/45] 10.222.0.45 () {58 vars in 1361 bytes} [Mon Aug 1 16:24:33 2022] PUT /identity/v3/domains/default/users/cc7787b19b7142fdb85a77e20df6ff3d/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:24:33.468363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575164e2-2c60-4ca2-8b67-d1244f1430f8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:33.508067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710b6fa-a4a4-4685-b4bd-75c7b7e80511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:33.508710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710b6fa-a4a4-4685-b4bd-75c7b7e80511 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:33.508998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710b6fa-a4a4-4685-b4bd-75c7b7e80511 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:33.509278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710b6fa-a4a4-4685-b4bd-75c7b7e80511 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:33.509985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710b6fa-a4a4-4685-b4bd-75c7b7e80511 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:33.511471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710b6fa-a4a4-4685-b4bd-75c7b7e80511 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:33.544534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 20/46] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:24:33 2022] POST /identity/v3/projects => generated 312 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:24:35.390901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f09688-5fe7-4aa4-b948-1fededcba574 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.391424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f09688-5fe7-4aa4-b948-1fededcba574 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.391506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f09688-5fe7-4aa4-b948-1fededcba574 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.392969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 27/47] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:35.398567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d9879-1901-4d23-8483-b0035a51f71c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.399004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d9879-1901-4d23-8483-b0035a51f71c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.399251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d9879-1901-4d23-8483-b0035a51f71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.418785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cd2d9879-1901-4d23-8483-b0035a51f71c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:35.420027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cd2d9879-1901-4d23-8483-b0035a51f71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:35.452080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:35.493066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 21/48] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:35.516583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f09688-5fe7-4aa4-b948-1fededcba574 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:35.552416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118fc7-9d09-4960-b03b-a4f089f49447 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.552689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118fc7-9d09-4960-b03b-a4f089f49447 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.553081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118fc7-9d09-4960-b03b-a4f089f49447 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.554460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 22/49] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:35.559557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.559877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.560152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.578556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:35.579754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:35.624344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f09688-5fe7-4aa4-b948-1fededcba574 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:35.649017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 23/50] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:35.657477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:35.658119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.658396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.658667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.670155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:35.672116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:35.675382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:35.676510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:35.677274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 28/51] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:35.682395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:35.731327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:35.773901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:35.778129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930a72-a924-4c4c-bf03-c3aed2c57a3c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:35.786853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b0eb9-76b2-469f-b3d5-e5459aaf7eef None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:35.818983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:35.819639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.819948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.820214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.821385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:35.822885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:35.823082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:35.823716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.824031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.824308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.824505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 24/52] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:35.825124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:35.825435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:35.826905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:35.830089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:35.836222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 29/53] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:35.841768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:35.933661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d40c-1291-4d48-b9a5-8f5e22a02267 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:35.942231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975609ca-289a-4e38-b5d7-7158c764e857 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:35.949147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:35.971197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:35.971949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.972235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.972452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.973167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:35.974693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:35.981118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:35.982035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:35.982322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:35.982598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:35.983915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:35.985392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:35.987028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 30/54] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:35 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:35.993447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:36.006299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 25/55] 10.222.0.45 () {60 vars in 1140 bytes} [Mon Aug 1 16:24:35 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:24:36.128197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6462c06-4a21-4390-9d33-b05204ad4a25 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:36.168792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245f370-e023-4ab8-9ea5-515ad8a04de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:36.169460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f370-e023-4ab8-9ea5-515ad8a04de3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:36.169737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f370-e023-4ab8-9ea5-515ad8a04de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:36.170011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245f370-e023-4ab8-9ea5-515ad8a04de3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:36.171209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245f370-e023-4ab8-9ea5-515ad8a04de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:36.172733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245f370-e023-4ab8-9ea5-515ad8a04de3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:36.174416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 31/56] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:36.180881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:36.202182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:36.283800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7094e3d-2c14-4034-8f0d-ba97406fa3a6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:36.320304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:36.320938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:36.321223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:36.321500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:36.322304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.domain.id=default, user.id=cc7787b19b7142fdb85a77e20df6ff3d) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:36.322615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:36.324104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dbcc7-c717-456a-aa18-bb7bc13ca02d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:36.331362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 26/57] 10.222.0.45 () {56 vars in 1322 bytes} [Mon Aug 1 16:24:36 2022] GET /identity/v3/role_assignments?user.id=cc7787b19b7142fdb85a77e20df6ff3d&scope.domain.id=default&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:38.022507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ebe1-c924-4b37-9dde-33dbb3d21b41 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:38.023039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ebe1-c924-4b37-9dde-33dbb3d21b41 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:38.023039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ebe1-c924-4b37-9dde-33dbb3d21b41 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:38.024521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 32/58] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:38.030054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:38.030470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:38.030614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:38.050507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:38.051741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:38.124772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 27/59] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:38.149919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ebe1-c924-4b37-9dde-33dbb3d21b41 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:38.198700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:38.258387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ebe1-c924-4b37-9dde-33dbb3d21b41 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:38.294735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a72ef-dfd0-4e95-b7b5-d88502098e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:38.295302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a72ef-dfd0-4e95-b7b5-d88502098e8c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:38.295579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a72ef-dfd0-4e95-b7b5-d88502098e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:38.295882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a72ef-dfd0-4e95-b7b5-d88502098e8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:38.297146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a72ef-dfd0-4e95-b7b5-d88502098e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:38.298625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a72ef-dfd0-4e95-b7b5-d88502098e8c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:38.300379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 33/60] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:38.306338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:38.321737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:38.413788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5355f-88d6-460c-aed9-9fe58520bf35 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:38.451351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d1548-8a6f-4234-939f-d5c9a75db780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:38.452022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d1548-8a6f-4234-939f-d5c9a75db780 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:38.452305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d1548-8a6f-4234-939f-d5c9a75db780 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:38.452576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d1548-8a6f-4234-939f-d5c9a75db780 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:38.453278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d1548-8a6f-4234-939f-d5c9a75db780 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:38.454754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d1548-8a6f-4234-939f-d5c9a75db780 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:38.484446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 28/61] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:24:38 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:41.713786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0ba05-e165-412b-8830-1ac969a4857a None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:41.714798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0ba05-e165-412b-8830-1ac969a4857a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:41.715276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0ba05-e165-412b-8830-1ac969a4857a None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:41.716892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 34/62] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:41.722966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277f694-4a68-4399-affc-0704e879dc0a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:41.723915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277f694-4a68-4399-affc-0704e879dc0a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:41.724432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277f694-4a68-4399-affc-0704e879dc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:41.754298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4277f694-4a68-4399-affc-0704e879dc0a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:41.755903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4277f694-4a68-4399-affc-0704e879dc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:41.844200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:41.858244 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 29/63] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:41.907053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0ba05-e165-412b-8830-1ac969a4857a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:41.913600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03fd1-cc85-4347-801c-f4c989350782 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:41.913889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03fd1-cc85-4347-801c-f4c989350782 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:41.914331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03fd1-cc85-4347-801c-f4c989350782 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:41.915882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 30/64] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:24:41.921569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af149c30-605e-42e4-9659-865be7ec1cd2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:41.922231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af149c30-605e-42e4-9659-865be7ec1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:41.922645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af149c30-605e-42e4-9659-865be7ec1cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:41.944558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-af149c30-605e-42e4-9659-865be7ec1cd2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:41.946047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-af149c30-605e-42e4-9659-865be7ec1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:42.023909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 31/65] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:42.032521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fc493-2700-4924-8d1e-672d7419c38a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.033129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fc493-2700-4924-8d1e-672d7419c38a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.033586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fc493-2700-4924-8d1e-672d7419c38a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.035539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 32/66] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:42.042119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.042840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.043316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.057845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0ba05-e165-412b-8830-1ac969a4857a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.075426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.078168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:42.079862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:42.121454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.122643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.122995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.123954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.141281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:42.146989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:42.148000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:42.148776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:42.151554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 35/67] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:42.157394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.197038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 33/68] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 01 16:24:42.197882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.273131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.282584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb8e9-ee38-46a9-9c78-3b5676d9742a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.325151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d937b-f100-40f4-9afa-180c7a9689d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.325815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d937b-f100-40f4-9afa-180c7a9689d9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.326085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d937b-f100-40f4-9afa-180c7a9689d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.326363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d937b-f100-40f4-9afa-180c7a9689d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.327967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48d67-c1ac-4697-9b57-761c151e0c2e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.338752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d937b-f100-40f4-9afa-180c7a9689d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-510d937b-f100-40f4-9afa-180c7a9689d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:42.343746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:42.344620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 36/69] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:42.348971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693f8c5-7bb3-4872-8d06-080abc71f67f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.350249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693f8c5-7bb3-4872-8d06-080abc71f67f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.350249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693f8c5-7bb3-4872-8d06-080abc71f67f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.350249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 37/70] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:42.350249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693f8c5-7bb3-4872-8d06-080abc71f67f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.380857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.381497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.381783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.382065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.382832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:42.383138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:42.383315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.384588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f9275-371b-4db3-b867-2202aeb32820 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:42.394036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 34/71] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:42.395300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902043-a4d8-47b0-8e26-4c2cd7a34e28 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.395592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902043-a4d8-47b0-8e26-4c2cd7a34e28 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.395929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902043-a4d8-47b0-8e26-4c2cd7a34e28 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.399677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 35/72] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:42.399677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9902043-a4d8-47b0-8e26-4c2cd7a34e28 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.462036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693f8c5-7bb3-4872-8d06-080abc71f67f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.502335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11843894-15de-45e1-963b-265152a1aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.503092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843894-15de-45e1-963b-265152a1aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.503533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843894-15de-45e1-963b-265152a1aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.504038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843894-15de-45e1-963b-265152a1aeb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.513482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9902043-a4d8-47b0-8e26-4c2cd7a34e28 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.514672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11843894-15de-45e1-963b-265152a1aeb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-11843894-15de-45e1-963b-265152a1aeb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:42.520186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:42.521122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:42.521877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 38/73] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:42.522852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1f59f-d68c-4739-958b-5d4e3cdd88dd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.522852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1f59f-d68c-4739-958b-5d4e3cdd88dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.523143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1f59f-d68c-4739-958b-5d4e3cdd88dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.537680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.542315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-01a1f59f-d68c-4739-958b-5d4e3cdd88dd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:42.543565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-01a1f59f-d68c-4739-958b-5d4e3cdd88dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:42.553910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.554557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.554838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.555108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.555875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:42.556177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:42.557592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:42.567214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 36/74] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:42.568145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.613810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 39/75] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:42.613810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.613810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.613810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.633700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:42.634903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:42.650742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.681165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7b34e-e3a3-4922-99e0-2055a0cfb7c7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.713223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 40/76] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:42.716808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.735701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.736765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.737204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.737771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.757824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:42.759331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:42.761150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 37/77] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:42.766652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.813913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.840215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df0a2f-198f-4f8d-ba74-9b5a6d69a101 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.875178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba2a73-bfe4-43b1-b30b-ab6e8c139b5c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:42.879922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.880569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.880853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.881131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.881863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:42.882171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:42.883595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:42.893093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 41/78] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:42.894017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:42.910762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:42.918326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:42.918983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:42.919266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:42.919538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:42.920781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:42.922259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:42.928909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 38/79] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:42.928909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.012664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e40db8-99a1-489d-8875-7f31f11c90dc None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.039543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26fad-cd60-4d70-8a67-5b252b3d0e43 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.049725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.052447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.053143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.053425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.053699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.065725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:43.071546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:43.072457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:43.073248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 42/80] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:43.073878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.080792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.081540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.081821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.082095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.083364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.084853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.086510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 39/81] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:43.091172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.156551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.187285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5759b8-2cd4-48a7-9571-a84ddacd8b53 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.200832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320acb91-6ef9-4ff1-a7cf-307b1788fd06 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.225749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.226743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.226743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.226938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.239320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.240012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.240296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.240567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.241795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.243260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.244913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 40/82] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:43.245660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.300778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.304375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.305442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.361796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b15f5-f7a6-4cfb-85a5-4b73e8568721 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.378842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.402634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.403005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.403286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.403558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.412463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 43/83] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:43.413464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.480628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.482152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.529694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec58fb-0398-43b0-81e8-3aac834f32d8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.531295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.564640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.565166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.565811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.566091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.566833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:43.567136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.568618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.577239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 44/84] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:43.578119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.595446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 41/85] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:43.596318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.617553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.695977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7ae6e-a859-4a32-8282-625d298aacf8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.712133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a28ea-2b17-4b8b-a0ff-483e200cccd5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.746190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.746765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.747156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.747314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.748045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.749518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.750576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.752301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.752301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.752301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.768805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.778084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 45/86] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:24:43 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:43.778971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.848215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.849755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.905771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:43.919023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670ae77-8ad4-499c-8907-58d257ec1914 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:43.961537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:43.963650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:43.963650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:43.963650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:43.964308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:43.965787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:43.967524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 46/87] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:43.969221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:43.989805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 42/88] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:43.990680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:44.050888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:44.088028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63c90-b16b-4976-bd4d-e12afaa97ce9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.130461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b571929-0c94-4940-946b-1ef99296e72f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.139233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.140290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.140790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.141292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.144352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.146170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.148180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 47/89] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:44.149191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:44.174149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.175544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.176184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.176481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.178352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.180755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.184069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 43/90] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:43 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:44.185241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:44.189931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:44.298871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778404ba-899b-40ad-a942-fcddcd583c22 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.334183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44d561-1ccf-4933-965e-9a9aef605b0c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.344112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.344927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.345304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.345664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.346596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=7803329d19ae4e269fa7a06c4a9d2eff) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:44.346998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.348662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.351723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:44.355209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 48/91] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:44 2022] GET /identity/v3/role_assignments?user.id=7803329d19ae4e269fa7a06c4a9d2eff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:44.356576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:44.384130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.384772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.385058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.385335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.387476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.389440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.392410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 44/92] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:44 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:44.393138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:44.545995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16133e8c-a0d6-4e0f-ae27-39409862e0f5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.572175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:44.585013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.585655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.585940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.586214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.587020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=cc7787b19b7142fdb85a77e20df6ff3d) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:44.587329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.588854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.590772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781f42d-415c-4dac-83a6-a3f342b5d540 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.595001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 45/93] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:44 2022] GET /identity/v3/role_assignments?user.id=cc7787b19b7142fdb85a77e20df6ff3d&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:44.604055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:44.667631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.668368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.668741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.669114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.741771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:44.837605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.839335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.870881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0d19-cd0a-4a5b-8ed9-b679c4b09607 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:44.942915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:44.943902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:44.944198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:44.944479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:44.945431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] RBAC: Adding query filter params (role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=7803329d19ae4e269fa7a06c4a9d2eff) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:44.946116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:44.948627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:44.957835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 46/94] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:44 2022] GET /identity/v3/role_assignments?user.id=7803329d19ae4e269fa7a06c4a9d2eff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 259 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:45.077831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:45.086936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 49/95] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:45.093168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:45.247729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da7c98-0877-463b-b94c-063ed73a1039 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:45.259401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:45.290999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbad82-ab29-45fe-90ba-a1113376a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:45.292061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbad82-ab29-45fe-90ba-a1113376a27f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:45.292563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbad82-ab29-45fe-90ba-a1113376a27f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:45.293035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbad82-ab29-45fe-90ba-a1113376a27f None admin] PATH_INFO: `/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:45.294433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbad82-ab29-45fe-90ba-a1113376a27f None admin] RBAC: Authorizing `identity:get_project(project_id=527b45db8b8e4266bd3b8b7a8d9332ec)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:45.296143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbad82-ab29-45fe-90ba-a1113376a27f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:45.298344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 47/96] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:45 2022] GET /identity/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:45.305782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:45.442790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981f84f-8269-4215-8385-f643bed15089 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:45.490899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:45.492017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:45.492491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:45.492953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:45.493945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=527b45db8b8e4266bd3b8b7a8d9332ec, user.id=7803329d19ae4e269fa7a06c4a9d2eff) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:45.494473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:45.496158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e1725-e5f1-4f90-be15-67373b3eb938 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:45.506414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 50/97] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:45 2022] GET /identity/v3/role_assignments?user.id=7803329d19ae4e269fa7a06c4a9d2eff&scope.project.id=527b45db8b8e4266bd3b8b7a8d9332ec&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:46.739265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff762df-8587-4c7b-b277-51379f3d234e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:46.740271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff762df-8587-4c7b-b277-51379f3d234e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:46.740777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff762df-8587-4c7b-b277-51379f3d234e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:46.742384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 48/98] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:24:46.748248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939f79-cb08-4e9a-b5ec-02c0476e6b8f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:46.749138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939f79-cb08-4e9a-b5ec-02c0476e6b8f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:46.749666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939f79-cb08-4e9a-b5ec-02c0476e6b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:46.770336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8a939f79-cb08-4e9a-b5ec-02c0476e6b8f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:46.771920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8a939f79-cb08-4e9a-b5ec-02c0476e6b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:46.826252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:46.844528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 51/99] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:46.866962 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff762df-8587-4c7b-b277-51379f3d234e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:46.914608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed570f9-e0e0-486c-8ec9-c6ef84eb9674 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:46.914930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed570f9-e0e0-486c-8ec9-c6ef84eb9674 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:46.915161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed570f9-e0e0-486c-8ec9-c6ef84eb9674 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:46.916709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 52/100] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:46.921594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:46.921873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:46.922147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:46.950762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:46.951905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:47.001498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff762df-8587-4c7b-b277-51379f3d234e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:47.055463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:47.057129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.057764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.059080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.060712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:47.063038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:47.073454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 49/101] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:47.075705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.077767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 53/102] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:47.121259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.142473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:47.202623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de5c69-8e00-4e24-b976-ce96d4a04a77 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:47.252503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:47.253466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.253975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.254236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.254948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:47.256464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:47.305559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 50/103] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:24:47 2022] POST /identity/v3/projects => generated 302 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:47.321310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbee21-b1f0-446b-b6b9-5d34bcb0077e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:47.348727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:47.375501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd33c21-6e1f-4881-8b05-79143368d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:47.376558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33c21-6e1f-4881-8b05-79143368d1ad None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.376836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33c21-6e1f-4881-8b05-79143368d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.377117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd33c21-6e1f-4881-8b05-79143368d1ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.398622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd33c21-6e1f-4881-8b05-79143368d1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-abd33c21-6e1f-4881-8b05-79143368d1ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:47.404353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:47.405267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:24:47.406026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:47.408953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 54/104] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:47.415148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.437620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefddc7-9948-4aea-be70-e52cc19f318b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.438255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefddc7-9948-4aea-be70-e52cc19f318b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.438730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefddc7-9948-4aea-be70-e52cc19f318b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.440964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 55/105] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:47.447650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.448402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.448874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.470083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:47.472253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:47.620070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 56/106] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:47.627608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:47.638147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdb65b-453e-4cc0-b446-38e9360ce0bd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:47.648860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.705114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:47.705773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.706562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.706562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.707267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:47.707623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:47.709174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:47.728125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 51/107] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:47.733938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.826500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f20839-7fec-412c-b142-55b9c78270d7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:47.840128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:47.862300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4987943-73d5-42bc-987d-4edc8fe9dfc5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:47.888959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:47.889628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.889909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.890183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.899526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:47.900223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:47.900508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:47.900782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] PATH_INFO: `/v3/users/22db5d288b9c4408ae9527c3854b9b19` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:47.914130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:47.915556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] RBAC: Authorizing `identity:get_user(user_id=22db5d288b9c4408ae9527c3854b9b19)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:47.917223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:47.918966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 52/108] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:47 2022] GET /identity/v3/users/22db5d288b9c4408ae9527c3854b9b19 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:47.919647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:47.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:47.921245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 57/109] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:47.924931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.927362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:47.947046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.035871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7f14-2764-4d68-8639-8a518ecca71b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.092673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5d4a-a177-4c38-a764-ecf0ba5abef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.093328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5d4a-a177-4c38-a764-ecf0ba5abef6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.093603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5d4a-a177-4c38-a764-ecf0ba5abef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.093870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5d4a-a177-4c38-a764-ecf0ba5abef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.116615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e180072-d777-414e-bfe2-ec567d15e75d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.199869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5d4a-a177-4c38-a764-ecf0ba5abef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:48.201402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5d4a-a177-4c38-a764-ecf0ba5abef6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:48.208166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.208815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.209099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.209378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.210120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:48.210427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:48.212613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b580ab-0bba-4d98-a9dd-a1f2e14e7380 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:48.224154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.236018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 58/110] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:48.237579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28707efc-d2e7-4136-8492-6ccd8d57c49c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.237868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28707efc-d2e7-4136-8492-6ccd8d57c49c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.238178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28707efc-d2e7-4136-8492-6ccd8d57c49c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.239807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 59/111] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:48.241518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e1d08-83ab-4180-b442-324af5bd69fd None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.242015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e1d08-83ab-4180-b442-324af5bd69fd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.242319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e1d08-83ab-4180-b442-324af5bd69fd None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.244766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 60/112] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:48.251013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e1d08-83ab-4180-b442-324af5bd69fd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:48.294800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.330918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 53/113] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:24:48.332388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c0764-a283-48e8-8a11-7f9ec6107f5c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.332682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c0764-a283-48e8-8a11-7f9ec6107f5c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.332961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c0764-a283-48e8-8a11-7f9ec6107f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.352725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e43c0764-a283-48e8-8a11-7f9ec6107f5c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:48.353951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e43c0764-a283-48e8-8a11-7f9ec6107f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:48.404671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e1d08-83ab-4180-b442-324af5bd69fd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.428197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 54/114] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:48.429313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.429602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.429882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.461919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:48.463131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:48.466168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.467985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.468370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.468659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.469993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:48.471626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:48.474535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 61/115] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:48 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:48.475322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:48.525360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.560970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 55/116] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:48.566497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:48.600167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.610239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7a8b4-a57c-4c1e-ae1d-6ac23ef7c819 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.665903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.666697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.670005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.670005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.670005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:48.673638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:48.673638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 62/117] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:48 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:48.681255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:48.719482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6689ca6-2de1-4c17-8d28-2909eaab7a4a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.760094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.760989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.761375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.761755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.775642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:48.782723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:48.784320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:24:48.785981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 56/118] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:48.791803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:48.815878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c393918-281a-4f46-9291-58a8453e6f7b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.845936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.855218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.856504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.856783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.857059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:48.858243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:48.859729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:48.861447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 63/119] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:48 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:48.866835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:48.907043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:48.942294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bd705-3b57-4d4d-9da7-37761e6f3acc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:48.984634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484abee7-5185-41cd-9181-90b2ee4bb40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:48.985294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abee7-5185-41cd-9181-90b2ee4bb40a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:48.985573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abee7-5185-41cd-9181-90b2ee4bb40a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:48.985872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484abee7-5185-41cd-9181-90b2ee4bb40a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.000394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484abee7-5185-41cd-9181-90b2ee4bb40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:49.003291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2f0f8-4c5f-47f6-a4d8-dffceb867d8e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-484abee7-5185-41cd-9181-90b2ee4bb40a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:49.006650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:49.007551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:24:49.008374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 57/120] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:24:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:49.009108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4523e-a895-400a-ad60-f9fa345f72d9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.009470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4523e-a895-400a-ad60-f9fa345f72d9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.009653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4523e-a895-400a-ad60-f9fa345f72d9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.011899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 58/121] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:49.011899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4523e-a895-400a-ad60-f9fa345f72d9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.060370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.061027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.061295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.061576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.062432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] RBAC: Adding query filter params (role.id=df635e925ad643e29aacbd1a1a32e0a2, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=22db5d288b9c4408ae9527c3854b9b19) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:49.062784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:49.064351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:49.072696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 64/122] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:48 2022] GET /identity/v3/role_assignments?user.id=22db5d288b9c4408ae9527c3854b9b19&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=df635e925ad643e29aacbd1a1a32e0a2 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:49.084855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.144865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:49.153906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4523e-a895-400a-ad60-f9fa345f72d9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.199721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.200425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.200754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.201030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.201793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:49.202088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:49.203570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:49.217010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 59/123] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:49.218096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.248476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:49.260849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf0ee6-d949-4a01-89ff-a107c8f7a2cf None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.306876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9cec1-475e-4256-8bc1-3d685660d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.307971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9cec1-475e-4256-8bc1-3d685660d123 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.308597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9cec1-475e-4256-8bc1-3d685660d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.309122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9cec1-475e-4256-8bc1-3d685660d123 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/7803329d19ae4e269fa7a06c4a9d2eff/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.311589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9cec1-475e-4256-8bc1-3d685660d123 None admin] RBAC: Authorizing `identity:create_grant(project_id=22cdac7ceb194d2c8b71fe379341ce04, user_id=7803329d19ae4e269fa7a06c4a9d2eff, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:49.313379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9cec1-475e-4256-8bc1-3d685660d123 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:49.330384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 65/124] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:24:49 2022] PUT /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/7803329d19ae4e269fa7a06c4a9d2eff/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:24:49.332861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.333712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.334272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.357307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:49.358957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:49.377071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c4839-909c-4658-9686-3fde2a3a5993 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.451303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.452100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.452386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.452657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.453418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:49.453726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:49.455303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:49.461974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 66/125] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:49.463237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.477293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 60/126] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:49.483143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.545212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:49.608683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:49.631273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef3292-1090-4530-8379-d5aae17d00e0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.646886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd76c0-5c1b-4317-a0c2-a8052cdb97b7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.678244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.683789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.683789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.683789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.683789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:49.683789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:49.685118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 67/127] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:49.686058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.719947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.720699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.720985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.721264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.722877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:49.724513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:49.726399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 61/128] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:49.727059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.842672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141e10-e9e7-4dc1-9ed4-7f9c308f8294 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.922583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:49.923199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:49.923414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:49.923684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:49.936575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:49.952118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9967c61-8c3b-4450-9808-7c2b0b491aa0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:49.953459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:49.958866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:49.959794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:49.960663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:49.967553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 68/129] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:49.968812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:49.996647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:50.006206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.006873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.007149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.007420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.011615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.013167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.016606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 62/130] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:50.017430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.132375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a0214-9fbf-45a4-ae34-93aee9bc636c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:50.152225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9200fbb-68e0-44a0-8970-be0798e29f50 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:50.170007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.170673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.170952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.171229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.172448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.173992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.176667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 69/131] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:49 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:50.177085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.199930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.201499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.202010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.202740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.203995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:50.204927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.212149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce85d4-bc2f-4049-9838-6063419229da None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.227316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 63/132] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:50.229689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482381-10e0-40e4-8255-0298155cf786 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.230195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482381-10e0-40e4-8255-0298155cf786 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.230679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482381-10e0-40e4-8255-0298155cf786 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.232766 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 64/133] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:50.234176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482381-10e0-40e4-8255-0298155cf786 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.265068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:50.296622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:50.324014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd249a2f-2b52-437d-939b-3766c6623deb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:50.371415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061848-40a5-45de-9fae-85519102be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.372798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061848-40a5-45de-9fae-85519102be15 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.373392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061848-40a5-45de-9fae-85519102be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.373862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061848-40a5-45de-9fae-85519102be15 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/cc7787b19b7142fdb85a77e20df6ff3d/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.376905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061848-40a5-45de-9fae-85519102be15 None admin] RBAC: Authorizing `identity:create_grant(project_id=22cdac7ceb194d2c8b71fe379341ce04, user_id=cc7787b19b7142fdb85a77e20df6ff3d, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.378924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061848-40a5-45de-9fae-85519102be15 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.398524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482381-10e0-40e4-8255-0298155cf786 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:50.403126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 70/134] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:24:50 2022] PUT /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/cc7787b19b7142fdb85a77e20df6ff3d/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:24:50.409741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061848-40a5-45de-9fae-85519102be15 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.467748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad7f76-c62c-48e2-884e-f951853acd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.468768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad7f76-c62c-48e2-884e-f951853acd37 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.468874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad7f76-c62c-48e2-884e-f951853acd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.469218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad7f76-c62c-48e2-884e-f951853acd37 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/7803329d19ae4e269fa7a06c4a9d2eff/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.475919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad7f76-c62c-48e2-884e-f951853acd37 None admin] RBAC: Authorizing `identity:create_grant(project_id=22cdac7ceb194d2c8b71fe379341ce04, user_id=7803329d19ae4e269fa7a06c4a9d2eff, role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.477752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad7f76-c62c-48e2-884e-f951853acd37 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.512884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 65/135] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:24:50 2022] PUT /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/7803329d19ae4e269fa7a06c4a9d2eff/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:24:50.514390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.514748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.515094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.542660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061848-40a5-45de-9fae-85519102be15 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:50.556379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:50.557507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:50.585381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.586596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.587066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.587508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.589223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.590956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.593176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 71/136] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:50 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:50.603249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.623298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:50.725370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 66/137] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:50.764311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.780682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa01c8f-db86-4985-9c67-2370c559b5a9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:50.782839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:50.818246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:50.818904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:50.819176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:50.819443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] PATH_INFO: `/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:50.820995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] RBAC: Authorizing `identity:get_project(project_id=527b45db8b8e4266bd3b8b7a8d9332ec)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:50.822488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:50.826565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 72/138] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:50 2022] GET /identity/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:50.835115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:50.962533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d984-c464-4679-86cc-9b1cafef5705 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:51.023102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:51.032770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:51.033500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:51.033783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:51.034098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:51.040840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:51.042395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:51.044404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 67/139] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:51.050363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:51.082587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4dc65-ff5a-40d9-88fd-a3f3284f357b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:51.151220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a116-866e-4b97-9e97-0124845226fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:51.151955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a116-866e-4b97-9e97-0124845226fa None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:51.152337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a116-866e-4b97-9e97-0124845226fa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:51.152620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a116-866e-4b97-9e97-0124845226fa None admin] PATH_INFO: `/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec/users/7803329d19ae4e269fa7a06c4a9d2eff/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:51.156357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402a116-866e-4b97-9e97-0124845226fa None admin] RBAC: Authorizing `identity:create_grant(project_id=527b45db8b8e4266bd3b8b7a8d9332ec, user_id=7803329d19ae4e269fa7a06c4a9d2eff, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:51.157847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402a116-866e-4b97-9e97-0124845226fa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:51.200608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 73/140] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:24:50 2022] PUT /identity/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec/users/7803329d19ae4e269fa7a06c4a9d2eff/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:24:51.208133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:51.248323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb3914-3f6f-4102-992d-12eb825812b8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:51.289415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f6296-c01a-4df2-9852-adddaec6c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:51.290501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6296-c01a-4df2-9852-adddaec6c35e None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:51.290991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6296-c01a-4df2-9852-adddaec6c35e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:51.291471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6296-c01a-4df2-9852-adddaec6c35e None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:51.292565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f6296-c01a-4df2-9852-adddaec6c35e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:51.294303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f6296-c01a-4df2-9852-adddaec6c35e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:51.345255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 68/141] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:24:51 2022] POST /identity/v3/users => generated 283 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:24:51.772435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9c085-f470-461f-9012-4e8184c86ff8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:51.772787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9c085-f470-461f-9012-4e8184c86ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:51.773000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9c085-f470-461f-9012-4e8184c86ff8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:51.774387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 74/142] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:51.780074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bfd69-a69c-4b49-8b76-e1ea3e223d36 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:51.780664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bfd69-a69c-4b49-8b76-e1ea3e223d36 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:51.781150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bfd69-a69c-4b49-8b76-e1ea3e223d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:51.805220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3f6bfd69-a69c-4b49-8b76-e1ea3e223d36 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:51.806800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3f6bfd69-a69c-4b49-8b76-e1ea3e223d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:51.901671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 69/143] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:51.925066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9c085-f470-461f-9012-4e8184c86ff8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.013228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33946d-e3a4-4d8f-9474-305f967fdb6b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.014894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33946d-e3a4-4d8f-9474-305f967fdb6b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.015640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33946d-e3a4-4d8f-9474-305f967fdb6b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.022899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 70/144] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:52.034625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.035586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.036341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.080677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9c085-f470-461f-9012-4e8184c86ff8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.084377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:52.086049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:52.121697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:52.122357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.122635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.122908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.126658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:52.139717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:52.145002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:52.145884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:52.147210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 75/145] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:52.152203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.200200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:52.217667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 71/146] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:52.253252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.274992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a715b-9e3f-41aa-9876-c8b0d1807bad None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.337038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:52.337693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.337970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.338398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.339559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:52.340006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:52.341966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:52.358547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 76/147] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:52.366074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.457252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c73bc-b4c4-42b3-b9b0-ec604584bce4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.474817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:52.538706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:52.541926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.542381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.542654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.551242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405d9d-aa2d-499e-95c0-50572842ff8a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.568839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:52.574880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:52.575732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:52.576557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:24:52.577864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 72/148] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:24:52.587395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.630778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:52.631451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.631734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.632037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] PATH_INFO: `/v3/users/22db5d288b9c4408ae9527c3854b9b19` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.636538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] RBAC: Authorizing `identity:get_user(user_id=22db5d288b9c4408ae9527c3854b9b19)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:52.638043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:52.639981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 77/149] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:52 2022] GET /identity/v3/users/22db5d288b9c4408ae9527c3854b9b19 => generated 289 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:52.645540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.655387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:52.726593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57d57-3b36-4f03-a370-d0d5513e97e8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.781572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:52.782641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.783142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.783606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.785462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:52.786026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:52.787667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:52.795903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6685dd-16ee-4780-b7ed-427447e3bac2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.802283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 73/150] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:52.808247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.823719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:52.838909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:52.838909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:52.838909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:52.838909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:52.840211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:52.841682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:52.844767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 78/151] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:52 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:52.850018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:52.955482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd12496-dc56-4480-b1dd-5f5b479cbc58 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:52.972410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:52.993305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d5134-6fa6-429b-872a-f13c77669c51 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:53.012515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:53.013214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.013495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.014017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.015019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:53.018583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:53.018583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 74/152] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:52 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:53.024313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:53.051789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:53.052474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.052755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.053031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/22db5d288b9c4408ae9527c3854b9b19/roles/df635e925ad643e29aacbd1a1a32e0a2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.055086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] RBAC: Authorizing `identity:create_grant(project_id=22cdac7ceb194d2c8b71fe379341ce04, user_id=22db5d288b9c4408ae9527c3854b9b19, role_id=df635e925ad643e29aacbd1a1a32e0a2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:53.061410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:53.088168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 79/153] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:24:52 2022] PUT /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/users/22db5d288b9c4408ae9527c3854b9b19/roles/df635e925ad643e29aacbd1a1a32e0a2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:24:53.197802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a511a0-dffc-4b8a-929e-55dc4c4db2e6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:53.256105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bee1a0-fa03-4d67-a6f1-206442a22fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:53.256868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bee1a0-fa03-4d67-a6f1-206442a22fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.257154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bee1a0-fa03-4d67-a6f1-206442a22fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.257429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bee1a0-fa03-4d67-a6f1-206442a22fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.325937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:53.344461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bee1a0-fa03-4d67-a6f1-206442a22fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:53.346000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bee1a0-fa03-4d67-a6f1-206442a22fd8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:53.483543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 75/154] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:53.489259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:53.594362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:53.651476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ceac1-1490-4c3e-91de-41766c00798c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:53.674027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b76488-4743-4d1d-9bc0-e372c97c6d5d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.674355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b76488-4743-4d1d-9bc0-e372c97c6d5d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.674649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b76488-4743-4d1d-9bc0-e372c97c6d5d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.679671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 76/155] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:53.698291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.698609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.698844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.717973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:53.718635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.718909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.719179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.723639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:53.725332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:53.727540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 80/156] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:53 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:53.733390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:53.733536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:53.734741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:53.898321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 77/157] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:24:53.927300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f8fa8-4941-45c5-9cbc-45ffb50bb3f8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:53.945771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:53.969210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:53.969849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:53.983860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:53.984599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:53.984876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:53.985145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:53.985949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=7803329d19ae4e269fa7a06c4a9d2eff) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:53.986251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:53.988205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:53.999372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 81/158] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:53 2022] GET /identity/v3/role_assignments?user.id=7803329d19ae4e269fa7a06c4a9d2eff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:54.109295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b53c9-7d9f-43b6-a7cf-75834dbdc8ce None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:54.159967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:54.160996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:54.161472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:54.161937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:54.179262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:54.184738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:54.185671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:54.186445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:24:54.188805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 78/159] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:54.195349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:54.324212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cfd26-e428-4a16-b2d3-ed3f607df0dc None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:54.363805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:54.364631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:54.364904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:54.365174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:54.365903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:54.366202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:54.367613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9939a7e-39d1-4f43-a8c3-2f065db6a6f4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:54.383811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 82/160] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:54.389622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:54.508803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:54.529474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee36f37-d854-4844-badb-abb0aa771458 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:54.529813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee36f37-d854-4844-badb-abb0aa771458 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:54.530020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee36f37-d854-4844-badb-abb0aa771458 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:54.531613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 83/161] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:54.541191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:54.541469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:54.541748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:54.546601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cffa94-0e32-47d3-9078-25bea2b4ebff None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:54.571741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:54.572986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:54.597516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:54.598189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:54.598466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:54.598737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] PATH_INFO: `/v3/users/cc7787b19b7142fdb85a77e20df6ff3d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:54.605119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] RBAC: Authorizing `identity:get_user(user_id=cc7787b19b7142fdb85a77e20df6ff3d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:54.606306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:54.608350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 79/162] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:54 2022] GET /identity/v3/users/cc7787b19b7142fdb85a77e20df6ff3d => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:54.614715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:54.696154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:54.744823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 84/163] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:54.777216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27763b3-67bb-4fa4-ac8b-b23ed5300f73 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:54.791782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:54.836595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50123a5c-c0d9-44d4-b1bc-a33d40bdd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:54.837248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123a5c-c0d9-44d4-b1bc-a33d40bdd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:54.837532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123a5c-c0d9-44d4-b1bc-a33d40bdd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:54.837805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123a5c-c0d9-44d4-b1bc-a33d40bdd7e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:54.865906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:54.945762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56caea-1311-4233-8e43-62b2ed337a6b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:54.956939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50123a5c-c0d9-44d4-b1bc-a33d40bdd7e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:54.958472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50123a5c-c0d9-44d4-b1bc-a33d40bdd7e2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:54.999739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:55.000821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.001102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.001367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.017466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:55.027853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:55.028736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:55.029496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:55.032194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 85/164] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:24:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:55.038563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:55.104153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:55.214814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 80/165] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:55.216276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace439f-f4e0-40be-a0ea-4cb1b821e73f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.216560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace439f-f4e0-40be-a0ea-4cb1b821e73f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.216832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace439f-f4e0-40be-a0ea-4cb1b821e73f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.224805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 81/166] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:55.225577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace439f-f4e0-40be-a0ea-4cb1b821e73f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:55.228261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ecb55-0ad9-49c9-84c4-4a46faa12757 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:55.317936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:55.318585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.318870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.319152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.319944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:55.320255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:55.321726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907fc5-a35b-4934-b702-b002217664e0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:55.328152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:55.342098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 86/167] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:24:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:55.343478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.344344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.344629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.382904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:55.384491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:55.485692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace439f-f4e0-40be-a0ea-4cb1b821e73f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:55.540052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:55.541071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.541581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.542041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.543441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:55.545426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:55.552078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 82/168] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:55 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:55.552245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 87/169] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:55.552324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:55.560848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:55.650439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:55.798890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a528a68-6eec-431c-bbaf-49c4a4ec35fe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:55.822676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d761-d12c-4f21-a121-6c7dbdc91c43 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:55.848377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:55.864726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:55.865388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.865668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.865940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.866760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=cc7787b19b7142fdb85a77e20df6ff3d) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:55.867070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:55.868660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:55.875174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 88/170] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:55 2022] GET /identity/v3/role_assignments?user.id=cc7787b19b7142fdb85a77e20df6ff3d&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:55.880818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:55.929201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:55.929989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:55.930281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:55.930568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:55.936563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:55.938446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:55.940918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 83/171] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:55 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:55.950060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:56.036947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60136c30-cd6e-4392-b6da-36cddafc3fe9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:56.062598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:56.092634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:56.093324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.093606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.093885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.112597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:56.118550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:56.119514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:56.120452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 89/172] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:56.128652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:56.256026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f699081-f4c2-4293-80e0-3d921784aa76 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:56.324237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee9fae-ada6-4486-baf6-7e63ab32ac03 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:56.326408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:56.327318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.327740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.328185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.394761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:56.421669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:56.423418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:56.432932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:56.433169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.433440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.433709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.434447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:56.434747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:56.436181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a2ed4-9aa3-4e50-b5c0-1f2b8229b084 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:56.451412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 90/173] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:56.453057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5168b-fbae-44da-b636-804946ca346b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.453337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5168b-fbae-44da-b636-804946ca346b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.453607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5168b-fbae-44da-b636-804946ca346b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.454914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 91/174] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:56.461672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.461947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.462219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.501246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:56.502551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:56.523896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:56.547657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 84/175] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:24:56.548546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:56.637381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:56.664845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09031c9b-1bc6-47df-8368-403507b1a62a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:56.691342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 92/176] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:56.696717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:56.706061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffa22b-96c7-49c2-9e17-f630c2acb6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:56.707019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffa22b-96c7-49c2-9e17-f630c2acb6fd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.707301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffa22b-96c7-49c2-9e17-f630c2acb6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.707576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffa22b-96c7-49c2-9e17-f630c2acb6fd None admin] PATH_INFO: `/v3/users/7803329d19ae4e269fa7a06c4a9d2eff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.709075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffa22b-96c7-49c2-9e17-f630c2acb6fd None admin] RBAC: Authorizing `identity:get_user(user_id=7803329d19ae4e269fa7a06c4a9d2eff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:56.710620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffa22b-96c7-49c2-9e17-f630c2acb6fd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:56.712521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 85/177] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:56 2022] GET /identity/v3/users/7803329d19ae4e269fa7a06c4a9d2eff => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:56.713595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cd84c-3a29-41c7-b968-b4d690cbadfe None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.713876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cd84c-3a29-41c7-b968-b4d690cbadfe None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.714147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cd84c-3a29-41c7-b968-b4d690cbadfe None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.720161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 86/178] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:56.720907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cd84c-3a29-41c7-b968-b4d690cbadfe None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:56.816714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791313a-35c4-402c-ae4a-1cfc9130b716 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:56.835162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cd84c-3a29-41c7-b968-b4d690cbadfe None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:56.852832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554d947-2b31-4cdf-b8e9-d3b8d5d89983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:56.853654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554d947-2b31-4cdf-b8e9-d3b8d5d89983 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.853933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554d947-2b31-4cdf-b8e9-d3b8d5d89983 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.854203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554d947-2b31-4cdf-b8e9-d3b8d5d89983 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.855372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554d947-2b31-4cdf-b8e9-d3b8d5d89983 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:56.857096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554d947-2b31-4cdf-b8e9-d3b8d5d89983 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:56.858934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 93/179] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:56 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:56.865299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.865573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.865837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.874559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6717cc5-bbee-4788-8723-34e8cfe70bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:56.875253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6717cc5-bbee-4788-8723-34e8cfe70bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:56.875890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6717cc5-bbee-4788-8723-34e8cfe70bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:56.876215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6717cc5-bbee-4788-8723-34e8cfe70bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:56.884845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:56.886414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:56.891843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:56.898391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:56.951073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6717cc5-bbee-4788-8723-34e8cfe70bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:56.952642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6717cc5-bbee-4788-8723-34e8cfe70bb9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:56.960620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 94/180] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:56.961468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.055826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 87/181] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:57.057099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770dc07-dfa9-403a-aaeb-3b3748d84c25 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.057387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770dc07-dfa9-403a-aaeb-3b3748d84c25 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.057659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770dc07-dfa9-403a-aaeb-3b3748d84c25 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.058927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 88/182] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:57.060564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770dc07-dfa9-403a-aaeb-3b3748d84c25 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.072558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422541-cc7b-4539-8596-5111b771e8e9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.106791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.108234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e16f-493e-44bd-8af4-14955d44eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.108931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e16f-493e-44bd-8af4-14955d44eeea None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.109250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e16f-493e-44bd-8af4-14955d44eeea None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.109514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e16f-493e-44bd-8af4-14955d44eeea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.120592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e16f-493e-44bd-8af4-14955d44eeea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f988e16f-493e-44bd-8af4-14955d44eeea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:57.125856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:57.126763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:57.127677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 95/183] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:24:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:57.129058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15211e94-d8c6-4ded-8f4e-28ba34d4f30f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.129334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15211e94-d8c6-4ded-8f4e-28ba34d4f30f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.129573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15211e94-d8c6-4ded-8f4e-28ba34d4f30f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.131029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 96/184] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:24:57.131543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15211e94-d8c6-4ded-8f4e-28ba34d4f30f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.135663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.173523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770dc07-dfa9-403a-aaeb-3b3748d84c25 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.211812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.212438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.212711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.212976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.213788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] RBAC: Adding query filter params (role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=7803329d19ae4e269fa7a06c4a9d2eff) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:57.214087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:57.215525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:57.221894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 89/185] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/role_assignments?user.id=7803329d19ae4e269fa7a06c4a9d2eff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:57.222987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.248479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15211e94-d8c6-4ded-8f4e-28ba34d4f30f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.290505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce20dd-f1a8-4db2-a19d-e4de1d87b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.291159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce20dd-f1a8-4db2-a19d-e4de1d87b558 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.291431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce20dd-f1a8-4db2-a19d-e4de1d87b558 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.291710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce20dd-f1a8-4db2-a19d-e4de1d87b558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.303421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce20dd-f1a8-4db2-a19d-e4de1d87b558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b2ce20dd-f1a8-4db2-a19d-e4de1d87b558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:57.309034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:57.310006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:57.310777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 97/186] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:57.317369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1235b2-9e96-4d02-b1bd-e46b13630825 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.317653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1235b2-9e96-4d02-b1bd-e46b13630825 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.317880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1235b2-9e96-4d02-b1bd-e46b13630825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.336922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6d1235b2-9e96-4d02-b1bd-e46b13630825 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:57.338054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6d1235b2-9e96-4d02-b1bd-e46b13630825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:57.343827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd5ea-438c-40e8-86ef-15a56aad6a72 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.355808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.388403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.389213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.390287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.390287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] PATH_INFO: `/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.391788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] RBAC: Authorizing `identity:get_project(project_id=527b45db8b8e4266bd3b8b7a8d9332ec)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:57.393532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:57.395292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 90/187] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/projects/527b45db8b8e4266bd3b8b7a8d9332ec => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:57.397364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.402611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.412349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 98/188] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:57.413762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.414050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.414357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.433067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:24:57.434265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:24:57.506852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989281e-6f71-46c6-8e43-b68b7c70c042 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.508397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 99/189] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:24:57.516030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.541040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.541590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.541873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.542138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.542851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:57.543149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:57.544572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:57.553704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 91/190] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:57.554510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.595826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.619730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0d824-6148-4101-b5fd-c2395a64a22c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.643367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.655908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.656505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddac91-50b8-4530-a2eb-e76f65d3f0be None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.656610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.656792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.657057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.657779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:57.658076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:57.659521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:57.667491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 100/191] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:57.668318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.699472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.700126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.700400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.700669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.701451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=527b45db8b8e4266bd3b8b7a8d9332ec, user.id=7803329d19ae4e269fa7a06c4a9d2eff) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:57.701754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:57.703206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:57.709231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 92/192] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/role_assignments?user.id=7803329d19ae4e269fa7a06c4a9d2eff&scope.project.id=527b45db8b8e4266bd3b8b7a8d9332ec&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:57.709994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.779233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031941-06cb-4617-9ca2-19e2b63d1eaf None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.829717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.830974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.830974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.831387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.844943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55913f-44f5-45c5-98b2-b66b93ee704e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:57.845207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:24:57.847327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:24:57.852331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:24:57.853293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:24:57.854079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 101/193] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:24:57.854749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.889097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:57.889097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:57.889413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:57.889651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:57.892536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:57.894571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:57.896310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 93/194] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:57.900913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:57.916209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:57.976408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4730d9c-d181-48ac-b62f-8231e6bff182 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.009864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.010565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.010692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00c0c2-1792-4e9c-bbba-95c4ca51c575 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.010818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.011122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] PATH_INFO: `/v3/users/576a56e85ce840639ae71fcd01b99cda` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.026354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] RBAC: Authorizing `identity:get_user(user_id=576a56e85ce840639ae71fcd01b99cda)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.027861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.029692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 102/195] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/users/576a56e85ce840639ae71fcd01b99cda => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:58.030284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.048281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.048931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.049208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.049591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] PATH_INFO: `/v3/users/576a56e85ce840639ae71fcd01b99cda` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.051113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] RBAC: Authorizing `identity:get_user(user_id=576a56e85ce840639ae71fcd01b99cda)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.052626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.054236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 94/196] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:57 2022] GET /identity/v3/users/576a56e85ce840639ae71fcd01b99cda => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:58.054973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.096395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.132195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887991f-bd57-4099-b685-606153f7ea61 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.134460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.156477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441acf7-a0ea-4ca6-80fb-4ae95b44b556 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.167552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.168186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.168456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.168725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.169470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:58.169767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.171198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.179514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 103/197] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:58.180570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.190516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.191291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.191572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.191877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.192569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.194033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.215390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 95/198] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:24:58 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:24:58.216436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.301495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd3c5c-3074-4b05-ab8a-912f4028fdf8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.327425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.337710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4f59-c696-4f03-afce-64b64ff877cd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.342724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.343410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.343656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.343960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.376820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.377343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.377547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.377958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.391162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.437550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.439716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.461703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.463706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.561815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 104/199] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:58.563081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.579410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.587680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 96/200] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:58.593794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.653650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.710289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2e02-e27a-437e-a449-c90da3a024d3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.717052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff85ac-950d-4b4e-a11a-0faeb4496699 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.755373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.756074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.756416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.756680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.758164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.759812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.760632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.761503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.761801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 97/201] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:58.762162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.762616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] PATH_INFO: `/v3/users/22db5d288b9c4408ae9527c3854b9b19` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.764040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] RBAC: Authorizing `identity:get_user(user_id=22db5d288b9c4408ae9527c3854b9b19)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.765754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.767416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.767885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 105/202] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/users/22db5d288b9c4408ae9527c3854b9b19 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:58.769127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.857546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.890912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e9939-0ffd-4635-8d48-b0d4abe82622 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.914999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aed75f-ed20-430b-bf27-00c678a1885c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:58.934397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.935201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.935483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.935779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.936747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=d8d875a7272446878db318488c144567, user.id=576a56e85ce840639ae71fcd01b99cda) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:58.937066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.940889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.943830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:58.949577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 106/203] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/role_assignments?user.id=576a56e85ce840639ae71fcd01b99cda&scope.project.id=d8d875a7272446878db318488c144567&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:58.950493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:58.971187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:58.972718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:58.973217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:58.973944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:58.976195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:58.978490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:58.985590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 98/204] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:58.993422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:59.190897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caeb53-9fb4-4e83-93e8-45cdbd0ff7d3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:59.222951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cecb37-7533-4c2f-81f7-bce61e75e578 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:59.230865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:59.231749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:59.232055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:59.232324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:59.260342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:59.270067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:59.271090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:59.271609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:59.272180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:59.273208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] RBAC: Adding query filter params (role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=d8d875a7272446878db318488c144567, user.id=576a56e85ce840639ae71fcd01b99cda) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:59.273795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:59.275517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:59.282595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 99/205] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/role_assignments?user.id=576a56e85ce840639ae71fcd01b99cda&scope.project.id=d8d875a7272446878db318488c144567&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:24:59.314690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:59.320078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:59.321660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:59.446469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 107/206] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:24:59.452421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:59.638967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9b06c-c169-4b83-be38-a232dcdb43bb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:59.696655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b92ce-db3a-49cd-802a-09e78a02fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:59.697737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b92ce-db3a-49cd-802a-09e78a02fc95 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:59.698205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b92ce-db3a-49cd-802a-09e78a02fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:59.698652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b92ce-db3a-49cd-802a-09e78a02fc95 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:59.700288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b92ce-db3a-49cd-802a-09e78a02fc95 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:59.702202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b92ce-db3a-49cd-802a-09e78a02fc95 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:59.704856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 100/207] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:24:59 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:24:59.717523 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:24:59.850942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:24:59.857662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f5a81-40d5-4408-b5c7-79da0d8d6423 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:24:59.913913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:24:59.914902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:24:59.915373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:24:59.915870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:24:59.916906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] RBAC: Adding query filter params (role.id=df635e925ad643e29aacbd1a1a32e0a2, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04, user.id=22db5d288b9c4408ae9527c3854b9b19) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:24:59.917410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:24:59.919042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41475c39-eda3-4519-bb32-9511d538ee27 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:24:59.928986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 108/208] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:24:59 2022] GET /identity/v3/role_assignments?user.id=22db5d288b9c4408ae9527c3854b9b19&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=df635e925ad643e29aacbd1a1a32e0a2 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:00.645808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a7fea-e4a8-425c-873a-7f54d92be40b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:00.646370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a7fea-e4a8-425c-873a-7f54d92be40b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:00.646370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a7fea-e4a8-425c-873a-7f54d92be40b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:00.647971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 101/209] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:00.653476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:00.653885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:00.654091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:00.669391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c549-cc82-480b-8084-c3f750e580bc None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:00.669690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c549-cc82-480b-8084-c3f750e580bc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:00.669982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c549-cc82-480b-8084-c3f750e580bc None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:00.671417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 102/210] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:00.676588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:00.678339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:00.679547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:00.680055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:00.680540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:00.699348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:00.700872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:00.765502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:00.765696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 109/211] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:00.779177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 103/212] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:00.791139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:00.804773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:00.899057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:00.911808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c99836-8f43-4876-95be-168c34d66b9e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:00.926063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bdaff-da74-4c34-ac7c-1f83d1a39cf1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:00.957796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:00.958575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:00.958850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:00.959117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:00.975548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:00.976226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:00.976572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:00.976814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:00.977622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:00.977960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:00.979424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:00.986013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 104/213] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:00.986139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:00.987045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:00.988009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:00.988618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:00.989247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 110/214] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:00.994676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:01.076912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:01.109019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bf1c4-6e4f-4a4c-8993-fedcaa31a3c6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:01.127178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0536a4-a29b-4a32-9f86-6302304411ce None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:01.157050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:01.158177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.159041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.159458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.160775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:01.161276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:01.163282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:01.163981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:01.164710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.165071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.165430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.166211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:01.167732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:01.173675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 111/215] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:01.179252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:01.203810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 105/216] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:00 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:01.219041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:01.307779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d1fe-7dea-4f64-87a0-0240cfca4c82 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:01.346773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecb879-5900-4b77-97f1-f958330d49d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:01.347560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecb879-5900-4b77-97f1-f958330d49d4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.347861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecb879-5900-4b77-97f1-f958330d49d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.348181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecb879-5900-4b77-97f1-f958330d49d4 None admin] PATH_INFO: `/v3/users/7337c043aee1472785d4ffea12ddc39a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.364734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecb879-5900-4b77-97f1-f958330d49d4 None admin] RBAC: Authorizing `identity:get_user(user_id=7337c043aee1472785d4ffea12ddc39a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:01.366286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecb879-5900-4b77-97f1-f958330d49d4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:01.368273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 112/217] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:01 2022] GET /identity/v3/users/7337c043aee1472785d4ffea12ddc39a => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:01.373935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:01.475700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6cb48-7af2-4981-bcfb-ed89c18fdf84 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.476048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6cb48-7af2-4981-bcfb-ed89c18fdf84 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.476767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6cb48-7af2-4981-bcfb-ed89c18fdf84 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.478590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 113/218] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:01.484069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.484357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.484625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.492254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368d887-8fd6-4a62-8e25-b1d2e0b4f5b1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:01.507894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:01.509436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:01.512708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:01.544035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:01.545304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.546205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.546980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.612715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 114/219] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:01.639689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:01.641638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:01.642217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:01.643425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:01.763894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6acfa1-efcf-4b59-8c43-6d82ce5e6aa4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:01.765875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 106/220] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:01.771982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:01.806333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:01.808527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:01.810102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:01.810617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:01.811113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:01.824572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:01.830212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:01.831194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:01.832054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:01.834790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 115/221] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:01.840891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:01.916823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:01.973688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86caef65-ead2-4e82-b6e3-2a7e3b2c929f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:01.974120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549a1ad-ac1d-4d4d-9dd3-9d601eef3c27 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.011596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e8420-69a3-4561-85f6-035126a1edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.011923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.012294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e8420-69a3-4561-85f6-035126a1edc1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.012445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.012566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e8420-69a3-4561-85f6-035126a1edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.012696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.012815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e8420-69a3-4561-85f6-035126a1edc1 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.012930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.013542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:02.013838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.014049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e8420-69a3-4561-85f6-035126a1edc1 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.015253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.015410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e8420-69a3-4561-85f6-035126a1edc1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.022316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 107/222] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:01 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:02.023862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f784f-a50e-402a-960a-6721737d70cb None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.024151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f784f-a50e-402a-960a-6721737d70cb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.024484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f784f-a50e-402a-960a-6721737d70cb None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.025416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 116/223] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:02.026146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 108/224] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:02.026812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.032930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.033267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.033637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.052977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:02.054291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:02.137350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 109/225] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:02.138246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.145814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe24bd-ed4d-4966-a152-7765ef243832 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.155357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:02.175632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:02.188802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.189830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.190978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.191431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.192491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=d8d875a7272446878db318488c144567, user.id=7337c043aee1472785d4ffea12ddc39a) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:02.192977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.194609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.200786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 117/226] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:02 2022] GET /identity/v3/role_assignments?user.id=7337c043aee1472785d4ffea12ddc39a&scope.project.id=d8d875a7272446878db318488c144567&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:02.202994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.254820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175ab9-9b4a-4234-86e2-af58c08438b6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.304241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.304241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.304241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.304577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] PATH_INFO: `/v3/users/576a56e85ce840639ae71fcd01b99cda` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.308127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=576a56e85ce840639ae71fcd01b99cda)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.310960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.314334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 110/227] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:02 2022] GET /identity/v3/users/576a56e85ce840639ae71fcd01b99cda => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:02.318770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.437510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:02.449798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8936-4e7d-4950-ac24-89b50f6a038b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.460765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61269b1-5635-47ee-9e0b-33af327c2aa4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.503078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.503743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.504040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.504310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.505677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.507146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.508878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 111/228] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:02 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:02.514454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.528200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.530123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.530403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.530673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.554970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:02.569915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:02.570838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:02.572214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 118/229] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:02.576896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.647788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2e329-f2d6-4318-91d7-0b1707aeb08e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.672009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:02.693457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.694236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.694519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.694791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567/users/576a56e85ce840639ae71fcd01b99cda/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.697052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d875a7272446878db318488c144567, user_id=576a56e85ce840639ae71fcd01b99cda, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.698535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.719567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 112/230] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:02 2022] PUT /identity/v3/projects/d8d875a7272446878db318488c144567/users/576a56e85ce840639ae71fcd01b99cda/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:02.773800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8894a4c-4a25-4c5a-a2c6-e5da25422b47 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:02.836279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:02.836836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:02.837113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:02.837374 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:02.838111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:02.838416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:02.839844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:02.855488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 119/231] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:02.861279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:02.884368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:03.011231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b5204-c241-4fe8-89e9-e7dec930ccb0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:03.056909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:03.057936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:03.058457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:03.058924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] PATH_INFO: `/v3/users/576a56e85ce840639ae71fcd01b99cda` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:03.060317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] RBAC: Authorizing `identity:get_user(user_id=576a56e85ce840639ae71fcd01b99cda)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:03.061969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:03.063805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 113/232] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:02 2022] GET /identity/v3/users/576a56e85ce840639ae71fcd01b99cda => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:03.069515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:03.204087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560281c7-47fd-4f72-8e9d-2ccf4c516f48 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:03.221591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:03.246353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276e351-fb98-450f-a6b7-8383928544e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:03.247000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276e351-fb98-450f-a6b7-8383928544e7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:03.247281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276e351-fb98-450f-a6b7-8383928544e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:03.247554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276e351-fb98-450f-a6b7-8383928544e7 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:03.248777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276e351-fb98-450f-a6b7-8383928544e7 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:03.250556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276e351-fb98-450f-a6b7-8383928544e7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:03.252733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 120/233] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:03 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:03.258380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:03.392141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:03.414392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2df0a-88d5-4fff-9bb5-d84f7abf6fac None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:03.455293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5530c-4d5a-4393-82d1-8dcbcbbab569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:03.456173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5530c-4d5a-4393-82d1-8dcbcbbab569 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:03.456458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5530c-4d5a-4393-82d1-8dcbcbbab569 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:03.456733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5530c-4d5a-4393-82d1-8dcbcbbab569 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567/users/576a56e85ce840639ae71fcd01b99cda/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:03.459218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5530c-4d5a-4393-82d1-8dcbcbbab569 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d875a7272446878db318488c144567, user_id=576a56e85ce840639ae71fcd01b99cda, role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:03.460754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5530c-4d5a-4393-82d1-8dcbcbbab569 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:03.490455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 114/234] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:03 2022] PUT /identity/v3/projects/d8d875a7272446878db318488c144567/users/576a56e85ce840639ae71fcd01b99cda/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:03.887210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf7dad-cd9c-4205-af8b-81e81293fbbc None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:03.887734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf7dad-cd9c-4205-af8b-81e81293fbbc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:03.887734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf7dad-cd9c-4205-af8b-81e81293fbbc None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:03.889344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 121/235] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:03.895101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfa86-441f-45ce-9ada-641c1a5d8752 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:03.895981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfa86-441f-45ce-9ada-641c1a5d8752 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:03.896449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dfa86-441f-45ce-9ada-641c1a5d8752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:03.916586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e07dfa86-441f-45ce-9ada-641c1a5d8752 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:03.918144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e07dfa86-441f-45ce-9ada-641c1a5d8752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:04.009867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 115/236] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:04.059965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf7dad-cd9c-4205-af8b-81e81293fbbc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:04.145311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433042-e1d4-418d-aee4-70c7a219fba8 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.145945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433042-e1d4-418d-aee4-70c7a219fba8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.146445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433042-e1d4-418d-aee4-70c7a219fba8 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.148079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 116/237] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:04.162066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.162769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.163236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.195377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:04.201093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:04.202388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf7dad-cd9c-4205-af8b-81e81293fbbc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:04.240117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:04.241136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.241607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.242069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.243542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:04.245372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:04.247420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 122/238] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:04.254102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:04.304747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:04.318936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:04.330948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 117/239] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:04.369982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9043b9-9f60-448d-90d8-7b17c4b9508a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:04.373968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:04.409848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:04.410823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.411320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.411818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.412704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:04.414368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:04.446733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 123/240] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:04 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:25:04.548064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999d59d-3ebe-45bc-9027-4ef157ac0198 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:04.604624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:04.605812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.606296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.606840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.624868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:04.631784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:04.632750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:04.633541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:04.636479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 118/241] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:04.644716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:04.769603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b987cd3-d9c3-4818-9054-c950474c7361 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:04.771443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:04.808421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:04.809077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.809410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.809679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.810468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:04.810775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:04.812286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f108f3-f417-4d91-805f-743b3b2056a5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:04.822571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 124/242] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:04.828423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:04.898528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:04.930787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e460-3c10-49af-81ec-0195b592e4c5 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.931285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e460-3c10-49af-81ec-0195b592e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.931285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e460-3c10-49af-81ec-0195b592e4c5 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.933675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 125/243] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:04.940363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.940831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.941310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:04.948618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdb2dd-eff0-4dac-b22e-cf2b15c9b7ce None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:04.969500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:04.970699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:04.991134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:04.992251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:04.992720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:04.993196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] PATH_INFO: `/v3/users/5685ae18c07a4b2cb46703b0e721e360` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.009760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] RBAC: Authorizing `identity:get_user(user_id=5685ae18c07a4b2cb46703b0e721e360)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:05.011686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:05.013681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 119/244] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:04 2022] GET /identity/v3/users/5685ae18c07a4b2cb46703b0e721e360 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:05.022067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.079902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 126/245] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:05.107271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.164057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471b18a-71a2-4600-8967-d0746741d670 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:05.189141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:05.216031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a4ca3-2dcf-4fa6-beca-3913f95bae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:05.217151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a4ca3-2dcf-4fa6-beca-3913f95bae2f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.217642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a4ca3-2dcf-4fa6-beca-3913f95bae2f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.218200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a4ca3-2dcf-4fa6-beca-3913f95bae2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.255871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:05.263139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebc0b5-6ea7-44bc-a25a-0055a9ad2792 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:05.304180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac0abc-7648-4a6b-9b27-91ab64bc4309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:05.304828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac0abc-7648-4a6b-9b27-91ab64bc4309 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.305106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac0abc-7648-4a6b-9b27-91ab64bc4309 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.305382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac0abc-7648-4a6b-9b27-91ab64bc4309 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.313267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a4ca3-2dcf-4fa6-beca-3913f95bae2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:05.316451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a4ca3-2dcf-4fa6-beca-3913f95bae2f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:05.318558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac0abc-7648-4a6b-9b27-91ab64bc4309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8eac0abc-7648-4a6b-9b27-91ab64bc4309 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:05.323596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:05.324521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:05.325386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:05.326506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 127/246] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:05.327068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12136b97-a797-460b-a146-3653b1e7b7f9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.327347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12136b97-a797-460b-a146-3653b1e7b7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.327623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12136b97-a797-460b-a146-3653b1e7b7f9 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.337191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 128/247] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:05.338651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12136b97-a797-460b-a146-3653b1e7b7f9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.454879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12136b97-a797-460b-a146-3653b1e7b7f9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:05.474822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 120/248] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:05.476419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.476847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.477247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.508141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:05.510391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:05.518221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:05.520421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.520421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.520618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.525174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:05.525487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:05.527871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:05.537564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:05.552077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 129/249] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:05.553043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.593241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:05.649494 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 121/250] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:05.650387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.776388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e31ccc-6f77-433b-a2c7-b558456bdc83 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:05.845576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:05.846707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.847338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.847845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.849933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:05.851836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:05.853936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01ddc9-d9b1-4106-b770-2d13b31f61dc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:05.858642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 130/251] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:05.859687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.917445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:05.918937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:05.919508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:05.920053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] PATH_INFO: `/v3/users/7337c043aee1472785d4ffea12ddc39a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:05.921535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=7337c043aee1472785d4ffea12ddc39a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:05.923482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:05.925645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 122/252] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/users/7337c043aee1472785d4ffea12ddc39a => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:05.926959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:05.962324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:05.978865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a2bbe-82e6-49bc-b28d-3b63f3a5a1ec None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:05.979436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:06.016048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.016986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.017427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.017936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.031263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:06.038361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:06.039501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:06.040520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:06.042880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 131/253] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:06.043846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:06.158922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9e1ce-9e1e-4917-ad2f-486978edc5b1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:06.199055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.200058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.200468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.200887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.202699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:06.204346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:06.206538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 132/254] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:06 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:06.207500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:06.224942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:06.254280 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5174c55-985f-46a1-bfa9-07fe7f229ac0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:06.300474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.301027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.301299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.301565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.302639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] RBAC: Adding query filter params (role.id=df635e925ad643e29aacbd1a1a32e0a2, scope.project.id=d8d875a7272446878db318488c144567, user.id=5685ae18c07a4b2cb46703b0e721e360) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:06.302639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:06.304493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:06.311220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 123/255] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:05 2022] GET /identity/v3/role_assignments?user.id=5685ae18c07a4b2cb46703b0e721e360&scope.project.id=d8d875a7272446878db318488c144567&role.id=df635e925ad643e29aacbd1a1a32e0a2 => generated 259 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:06.311879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:06.335321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4efc2-5951-4411-9d38-53ca52b9d72a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:06.383641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.384955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.385451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.386148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.387153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:06.387688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:06.390050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e062f-1c76-4e6a-95a7-95433a48612e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:06.402117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 133/256] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:06.403938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f1bfa-462f-44d5-a0fd-ed71fa54ec5c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.404418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f1bfa-462f-44d5-a0fd-ed71fa54ec5c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.404878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f1bfa-462f-44d5-a0fd-ed71fa54ec5c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.406456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 134/257] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:06.408057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f1bfa-462f-44d5-a0fd-ed71fa54ec5c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:06.456192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:06.501070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:06.523033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fac44-0736-4e3d-87aa-b89c8de08bc0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:06.553394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f1bfa-462f-44d5-a0fd-ed71fa54ec5c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:06.565380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79467e2-2e80-4fe7-8c1f-9ef4ffc30b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.566076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79467e2-2e80-4fe7-8c1f-9ef4ffc30b65 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.566356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79467e2-2e80-4fe7-8c1f-9ef4ffc30b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.566637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79467e2-2e80-4fe7-8c1f-9ef4ffc30b65 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567/users/7337c043aee1472785d4ffea12ddc39a/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.569105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79467e2-2e80-4fe7-8c1f-9ef4ffc30b65 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d875a7272446878db318488c144567, user_id=7337c043aee1472785d4ffea12ddc39a, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:06.570731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79467e2-2e80-4fe7-8c1f-9ef4ffc30b65 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:06.593674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 124/258] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:06 2022] PUT /identity/v3/projects/d8d875a7272446878db318488c144567/users/7337c043aee1472785d4ffea12ddc39a/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:06.596162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.597403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.598398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.598884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.599309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.599722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.600163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] PATH_INFO: `/v3/users/576a56e85ce840639ae71fcd01b99cda` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.603214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] RBAC: Authorizing `identity:get_user(user_id=576a56e85ce840639ae71fcd01b99cda)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:06.606571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:06.608924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 135/259] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:06 2022] GET /identity/v3/users/576a56e85ce840639ae71fcd01b99cda => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:06.615128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:06.638694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:06.640276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:06.764650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 125/260] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:06.805719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:06.862927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7860ff-7472-4c3f-b150-68096fea58a6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:06.932500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:06.933132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:06.933402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:06.933666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:06.937830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:06.960436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:06.992616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1794b-8d01-4cfd-8a15-622fd6e63644 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:07.052808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:07.054298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:07.073182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:07.073773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.074050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.074316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.094148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:07.099412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:07.100378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:07.101616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 126/261] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:07.121930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:07.186210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 136/262] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:07.192326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:07.248834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:07.254730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0aefa-4401-4692-93da-c1c1bdb6c63f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:07.282093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:07.290716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:07.291049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.291325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.291596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.292351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:07.292650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:07.294115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:07.303429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 127/263] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:07.311965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:07.344684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b325-0962-4bb1-8e78-412bdf68aea5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:07.428817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:07.430524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.430939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.431559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.437415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:07.438900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:07.441095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 137/264] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:07 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:07.447658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:07.556817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148f03-15b2-4eea-85d6-15216abb7b6d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:07.570356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:07.589719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2fc0f-48be-4c85-9011-f3b20c508dc6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:07.596686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:07.598538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.598974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.600227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.601169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=d8d875a7272446878db318488c144567, user.id=576a56e85ce840639ae71fcd01b99cda) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:07.601585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:07.603171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:07.614138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 138/265] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:07 2022] GET /identity/v3/role_assignments?user.id=576a56e85ce840639ae71fcd01b99cda&scope.project.id=d8d875a7272446878db318488c144567&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:07.641761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae2651e-b0af-437d-bdad-30c19b361570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:07.643391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2651e-b0af-437d-bdad-30c19b361570 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.643923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2651e-b0af-437d-bdad-30c19b361570 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.644339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2651e-b0af-437d-bdad-30c19b361570 None admin] PATH_INFO: `/v3/users/576a56e85ce840639ae71fcd01b99cda` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.646190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2651e-b0af-437d-bdad-30c19b361570 None admin] RBAC: Authorizing `identity:get_user(user_id=576a56e85ce840639ae71fcd01b99cda)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:07.647979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae2651e-b0af-437d-bdad-30c19b361570 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:07.650001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 128/266] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:07 2022] GET /identity/v3/users/576a56e85ce840639ae71fcd01b99cda => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:07.655842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:07.787227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259b0d-863b-431a-842d-ac017f6a2b49 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:07.825673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:07.826360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.826639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.826910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.867084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08849a6-a9e7-478b-a02e-8cbd2ea4035f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.867385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08849a6-a9e7-478b-a02e-8cbd2ea4035f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.867610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08849a6-a9e7-478b-a02e-8cbd2ea4035f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.869328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 129/267] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:07.876161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ccc6a-c07e-418e-a4be-124cff00bb05 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:07.876498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ccc6a-c07e-418e-a4be-124cff00bb05 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:07.876702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ccc6a-c07e-418e-a4be-124cff00bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:07.888359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:07.914550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-268ccc6a-c07e-418e-a4be-124cff00bb05 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:07.915799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-268ccc6a-c07e-418e-a4be-124cff00bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:07.917258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:07.918950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:07.989262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:08.006941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 130/268] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:08.008454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfafc5-dddf-49a4-ac3b-dda01367af38 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.008810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfafc5-dddf-49a4-ac3b-dda01367af38 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.009150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfafc5-dddf-49a4-ac3b-dda01367af38 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.010792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 131/269] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:08.016598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.016598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.016902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.040338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:08.041606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:08.052450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 139/270] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:08.054158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.130377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 132/271] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:08.131390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.164081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:08.198660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7d23e-57ef-46f0-95a8-ef38fc1cf31b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.239498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.240559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.241127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.241596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.254154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:08.259920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:08.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:08.261982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:08.264841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 140/272] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:08.264959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee00d6e-0a32-4515-8ed9-0253f1523484 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.266068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.284401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:08.316207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.316881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.317606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.317893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.319438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:08.321199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:08.327926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 133/273] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:08.330361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.389426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01431b-9fe7-4f23-8728-1e73dcfec237 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.435466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.437070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.437525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.437967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.440450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:08.442407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:08.445426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 141/274] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:08.446443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.450492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:08.470772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fbb80-f8ad-4988-a56a-c4a88c9bbb70 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.514126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.514796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.515073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.515345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.516116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:08.516423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:08.517953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:08.532995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 134/275] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:08.532995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.578753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff71590-5288-4242-8a66-50021a33e426 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.609864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:08.615568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.616487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.616995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.617458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.618440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] RBAC: Adding query filter params (role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=d8d875a7272446878db318488c144567, user.id=576a56e85ce840639ae71fcd01b99cda) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:08.618956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:08.620629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:08.634804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 142/276] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/role_assignments?user.id=576a56e85ce840639ae71fcd01b99cda&scope.project.id=d8d875a7272446878db318488c144567&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:08.636172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:08.726035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62ec7d-c5a9-4fe7-a71d-de390881c70a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.761158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:08.802107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945cfeb-21f5-4f97-8571-066c33f855a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.803628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cfeb-21f5-4f97-8571-066c33f855a5 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.803628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cfeb-21f5-4f97-8571-066c33f855a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.804147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cfeb-21f5-4f97-8571-066c33f855a5 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.805399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945cfeb-21f5-4f97-8571-066c33f855a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:08.808180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945cfeb-21f5-4f97-8571-066c33f855a5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:08.817315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb7e64-90fc-484e-89c1-8a51d0416c20 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:08.859352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:08.860498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:08.860973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:08.861468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] PATH_INFO: `/v3/users/04873ce505ca4ccab13c6bc6e4e32012` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:08.878349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] RBAC: Authorizing `identity:get_user(user_id=04873ce505ca4ccab13c6bc6e4e32012)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:08.880299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:08.882303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 143/277] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/users/04873ce505ca4ccab13c6bc6e4e32012 => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:08.885079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 135/278] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:08 2022] POST /identity/v3/users => generated 293 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:08.891314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:09.010511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd16a3-f927-4943-8361-a4a6c0d927c4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:09.048955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:09.060126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:09.061652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.062125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.062620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.063613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, user.id=04873ce505ca4ccab13c6bc6e4e32012, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:09.064159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:09.065977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9937b1b-6193-4145-8a91-ee5663fb6eac None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:09.072547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 144/279] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 16:25:08 2022] GET /identity/v3/role_assignments?user.id=04873ce505ca4ccab13c6bc6e4e32012&scope.system=all&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:09.288105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb171b7-9b28-4249-a8ec-b3614007f194 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.288616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb171b7-9b28-4249-a8ec-b3614007f194 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.288616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb171b7-9b28-4249-a8ec-b3614007f194 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.290297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 136/280] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:09.295884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.296164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.296447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.317351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:09.318572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:09.358926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f4a12-1383-456a-a5c5-c65767219346 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.359225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f4a12-1383-456a-a5c5-c65767219346 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.359459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f4a12-1383-456a-a5c5-c65767219346 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.360880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 137/281] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:09.368977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.369255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.369565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.395519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:09.396761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:09.397454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 145/282] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:09.423402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:09.520229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 138/283] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:09.556056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:09.563674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58517-e3a0-4874-aa49-eccae3aff01d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:09.592281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:09.613553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:09.614209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.614483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.614750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.629801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:09.642308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:09.644119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:09.644930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:09.645668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:09.652195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 146/284] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:09.677071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:09.776018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e41d2-ad26-4915-8da4-659b31705e52 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:09.844980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:09.845561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.845871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.846147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.858829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019331e-59fb-440e-bf54-f2bb2b651a55 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:09.871510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:09.877220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:09.878829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:09.879620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 139/285] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:09.887690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:09.949496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:09.950138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:09.950416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:09.950689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:09.951428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:09.951732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:09.953285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:09.974297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 147/286] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:09.985315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:10.003281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:10.010471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40b4d-d5a2-46fc-8089-ccd1b30784a4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:10.051798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:10.051798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:10.051798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:10.051798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:10.052206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:10.052206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:10.052533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:10.062456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 140/287] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:10.069292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:10.090642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:10.200774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe376452-b257-4488-ab6b-d943dc306c2e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:10.263971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:10.265000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:10.265504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:10.266695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] PATH_INFO: `/v3/users/5685ae18c07a4b2cb46703b0e721e360` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:10.269494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] RBAC: Authorizing `identity:get_user(user_id=5685ae18c07a4b2cb46703b0e721e360)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:10.271194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:10.275169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf6fbc-47fa-4fbc-9d45-b774d5f4c67b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:10.277255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 148/288] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:09 2022] GET /identity/v3/users/5685ae18c07a4b2cb46703b0e721e360 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:10.279154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:10.346971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:10.348394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:10.353139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:10.353674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] PATH_INFO: `/v3/users/7337c043aee1472785d4ffea12ddc39a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:10.356942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] RBAC: Authorizing `identity:get_user(user_id=7337c043aee1472785d4ffea12ddc39a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:10.358676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:10.360903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 141/289] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:10 2022] GET /identity/v3/users/7337c043aee1472785d4ffea12ddc39a => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:10.368843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:10.416105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:10.480716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcbe51-cdd0-41cd-8915-7391854a0a61 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:10.501376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:10.529736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:10.530384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:10.530655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:10.530934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:10.532139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:10.533742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:10.535701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 149/290] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:10 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:10.541151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:10.561779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa238-7433-426f-9ec1-248570d93f92 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:10.648624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94493249-12b4-45ea-8ea4-4fb962ae7a40 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:10.688038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:10.688910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:10.689450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:10.689913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567/users/5685ae18c07a4b2cb46703b0e721e360/roles/df635e925ad643e29aacbd1a1a32e0a2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:10.692083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d875a7272446878db318488c144567, user_id=5685ae18c07a4b2cb46703b0e721e360, role_id=df635e925ad643e29aacbd1a1a32e0a2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:10.694845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:10.697915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:10.698559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:10.699175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:10.699175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:10.719176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 150/291] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:10 2022] PUT /identity/v3/projects/d8d875a7272446878db318488c144567/users/5685ae18c07a4b2cb46703b0e721e360/roles/df635e925ad643e29aacbd1a1a32e0a2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:25:10.795520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:10.797319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:10.830248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:10.941363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 142/292] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:10.947538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:10.988287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:11.063406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409b96-708b-47dc-a36a-7afda188ff5e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:11.102363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809287-a057-4dc7-872e-c9ab5a8b4ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:11.103989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809287-a057-4dc7-872e-c9ab5a8b4ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.104461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809287-a057-4dc7-872e-c9ab5a8b4ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.104921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809287-a057-4dc7-872e-c9ab5a8b4ba3 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.110612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809287-a057-4dc7-872e-c9ab5a8b4ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:11.112522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809287-a057-4dc7-872e-c9ab5a8b4ba3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:11.114479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 151/293] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:10 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:11.120664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:11.271226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe92fb-1803-424f-a1c0-f825c5019fa2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:11.308826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:11.317259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74f7c3-82d8-4ca0-9781-c74021c79b35 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.317532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74f7c3-82d8-4ca0-9781-c74021c79b35 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.317740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74f7c3-82d8-4ca0-9781-c74021c79b35 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.324396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 152/294] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:11.325477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:11.326000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e26b40-ea57-463f-8274-7489e838435d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.326105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.326338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e26b40-ea57-463f-8274-7489e838435d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.326436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.326611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e26b40-ea57-463f-8274-7489e838435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.326715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.327406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=d8d875a7272446878db318488c144567, user.id=7337c043aee1472785d4ffea12ddc39a) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:11.327746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:11.329334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:11.343246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 143/295] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:11 2022] GET /identity/v3/role_assignments?user.id=7337c043aee1472785d4ffea12ddc39a&scope.project.id=d8d875a7272446878db318488c144567&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:11.356277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e4e26b40-ea57-463f-8274-7489e838435d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:11.357507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e4e26b40-ea57-463f-8274-7489e838435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:11.465949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 153/296] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:11.490273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:11.589119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d08e5-fb53-4fbe-9606-daf33f40aaa3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.589735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d08e5-fb53-4fbe-9606-daf33f40aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.590201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d08e5-fb53-4fbe-9606-daf33f40aaa3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.593171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 154/297] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:11.598209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.598781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.599267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.613526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba2f10-d0d1-4c51-961e-ce4fd237bc64 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:11.621083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:11.622577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:11.653614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:11.654287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.654583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.654822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.657418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:11.680108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:11.686322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:11.687261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:11.688126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:11.691124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 144/298] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:11.700686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:11.718491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 155/299] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:11.747913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:11.765776 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:11.836717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38f297-9495-4830-9322-a09f81be9359 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:11.881603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:11.882573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.883085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.883549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.884514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:11.885031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:11.886633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:11.896893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 145/300] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:11.901993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe746d-794b-4983-a3e8-b91f30fd2ffc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:11.902921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:11.944283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:11.945295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:11.945795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:11.946372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:11.958862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:11.964721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:11.965621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:11.966408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:11.969120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 156/301] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:11.975840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:12.011451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:12.036667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daffe6-9727-4a43-bbef-963624bfffa2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.047959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:12.072498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843fece-e604-4f60-b688-c601b443c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.073172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843fece-e604-4f60-b688-c601b443c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.073439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843fece-e604-4f60-b688-c601b443c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.073708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843fece-e604-4f60-b688-c601b443c0f2 None admin] PATH_INFO: `/v3/users/04873ce505ca4ccab13c6bc6e4e32012` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.074950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843fece-e604-4f60-b688-c601b443c0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=04873ce505ca4ccab13c6bc6e4e32012)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.076486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843fece-e604-4f60-b688-c601b443c0f2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.078121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 146/302] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:11 2022] GET /identity/v3/users/04873ce505ca4ccab13c6bc6e4e32012 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:12.084482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4291-86e2-473a-b2d8-46578e69dfee None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.084760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4291-86e2-473a-b2d8-46578e69dfee None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.085036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4291-86e2-473a-b2d8-46578e69dfee None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.086365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 147/303] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:12.088137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa4291-86e2-473a-b2d8-46578e69dfee None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:12.099399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab79d-a255-45a0-84d6-61aa4dd116c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.135853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.136469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.136665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.136935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.137663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:12.137960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.139346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bb0f9-b338-44d8-850f-1122544f21a3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.151872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 157/304] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:12.153505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.153909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.154301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.175696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:12.176960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:12.212785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa4291-86e2-473a-b2d8-46578e69dfee None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.252377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.252787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 158/305] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:12.253170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:12.260319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.260880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.261857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] PATH_INFO: `/v3/system/users/04873ce505ca4ccab13c6bc6e4e32012/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.264198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=04873ce505ca4ccab13c6bc6e4e32012, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.266548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.271736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:12.281547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 148/306] 10.222.0.45 () {58 vars in 1334 bytes} [Mon Aug 1 16:25:12 2022] PUT /identity/v3/system/users/04873ce505ca4ccab13c6bc6e4e32012/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:12.282788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:12.349998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:12.389529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5768ec-9e83-4df2-aa13-74ddbe0c3bfa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.420144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd6c26-8b1c-419a-a7db-963107645d01 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.429279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.430194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.430533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.431334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] PATH_INFO: `/v3/users/887844f9dad74652af24236883a6a757` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.448059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] RBAC: Authorizing `identity:get_user(user_id=887844f9dad74652af24236883a6a757)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.449547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.451576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 159/307] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:12 2022] GET /identity/v3/users/887844f9dad74652af24236883a6a757 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:12.458087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:12.470262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.471031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.471468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.472012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.473879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.475317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.477021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 149/308] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:12.489153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:12.557638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:12.593877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f330e1-7d0a-416d-96ae-592b4b4fd756 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.637566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058d4e-2ec9-4d75-bf30-5a7ea56b63a1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:12.643110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.644016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.644454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.644862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.645798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] RBAC: Adding query filter params (role.id=df635e925ad643e29aacbd1a1a32e0a2, user.id=887844f9dad74652af24236883a6a757, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:12.646308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.647940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cca5cc-ff5e-4691-9ddf-6d0f08b2e2fc None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.650717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:12.655394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 160/309] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 16:25:12 2022] GET /identity/v3/role_assignments?user.id=887844f9dad74652af24236883a6a757&scope.system=all&role.id=df635e925ad643e29aacbd1a1a32e0a2 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:12.692689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc628a68-35c6-4965-bc28-9fdfdaca4dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:12.693350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc628a68-35c6-4965-bc28-9fdfdaca4dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:12.693630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc628a68-35c6-4965-bc28-9fdfdaca4dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:12.693899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc628a68-35c6-4965-bc28-9fdfdaca4dd0 None admin] PATH_INFO: `/v3/groups` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:12.694584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc628a68-35c6-4965-bc28-9fdfdaca4dd0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:12.696285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc628a68-35c6-4965-bc28-9fdfdaca4dd0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:12.715718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 150/310] 10.222.0.45 () {60 vars in 1134 bytes} [Mon Aug 1 16:25:12 2022] POST /identity/v3/groups => generated 229 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:13.096194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8f32d-6712-4df6-9b82-ac8d14f522ea None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:13.096484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8f32d-6712-4df6-9b82-ac8d14f522ea None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:13.096786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8f32d-6712-4df6-9b82-ac8d14f522ea None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:13.098344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 161/311] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:13.104031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:13.104302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:13.104582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:13.123232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:13.124447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:13.197014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 151/312] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:13.219204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8f32d-6712-4df6-9b82-ac8d14f522ea None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:13.324262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8f32d-6712-4df6-9b82-ac8d14f522ea None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:13.360178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:13.361044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:13.361517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:13.361946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:13.370384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:13.373520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:13.381260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:13.382650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:13.383736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 162/313] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:13.387460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:13.489360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:13.493268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b324d-f4b8-4403-9d73-7dbd67d3930c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:13.534278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:13.534910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:13.535189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:13.535457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:13.536222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:13.536527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:13.537924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:13.547165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 152/314] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:13.552793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:13.658881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a2e96-6df6-4561-ad44-1271af4c0798 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:13.699292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:13.700074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:13.700382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:13.700654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] PATH_INFO: `/v3/users/5685ae18c07a4b2cb46703b0e721e360` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:13.701819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] RBAC: Authorizing `identity:get_user(user_id=5685ae18c07a4b2cb46703b0e721e360)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:13.703252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:13.705345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 163/315] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:13 2022] GET /identity/v3/users/5685ae18c07a4b2cb46703b0e721e360 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:13.710849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:13.817131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd699-a27b-4443-acda-d3e37b1871af None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:13.852894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:13.853853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:13.854320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:13.854763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:13.890359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:13.933578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:13.934972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:14.038043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 153/316] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:14.043827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:14.116019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:14.152675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f94d81-1b8b-46c1-90bf-386487ce53a9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:14.187508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cd7a7-b36e-448e-accb-8c7abae7aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:14.188103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cd7a7-b36e-448e-accb-8c7abae7aba0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.188379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cd7a7-b36e-448e-accb-8c7abae7aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.188654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cd7a7-b36e-448e-accb-8c7abae7aba0 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.189984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cd7a7-b36e-448e-accb-8c7abae7aba0 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:14.191482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cd7a7-b36e-448e-accb-8c7abae7aba0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:14.193545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 164/317] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:14 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:14.199237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:14.263702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:14.308394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0aef71-2ae1-4adf-98ec-557e43e04f7e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:14.350262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:14.350906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.351184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.351455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.352317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] RBAC: Adding query filter params (role.id=df635e925ad643e29aacbd1a1a32e0a2, scope.project.id=d8d875a7272446878db318488c144567, user.id=5685ae18c07a4b2cb46703b0e721e360) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:14.352628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:14.354125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb9d06-734a-4a63-9a46-148e5a5ed9bf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:14.361390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 154/318] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:14 2022] GET /identity/v3/role_assignments?user.id=5685ae18c07a4b2cb46703b0e721e360&scope.project.id=d8d875a7272446878db318488c144567&role.id=df635e925ad643e29aacbd1a1a32e0a2 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:14.468404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d20687-a658-4dc8-bbac-3a886670e470 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.468918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d20687-a658-4dc8-bbac-3a886670e470 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.468918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d20687-a658-4dc8-bbac-3a886670e470 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.470866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 165/319] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:14.477194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.477479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.477847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.498199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:14.499618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:14.583943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 155/320] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:14.605823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d20687-a658-4dc8-bbac-3a886670e470 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:14.732406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d20687-a658-4dc8-bbac-3a886670e470 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:14.771128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd272b-f5de-47a0-86c0-99acaa3e7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:14.772175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd272b-f5de-47a0-86c0-99acaa3e7438 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.772657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd272b-f5de-47a0-86c0-99acaa3e7438 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.773124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd272b-f5de-47a0-86c0-99acaa3e7438 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.775387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:14.786870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd272b-f5de-47a0-86c0-99acaa3e7438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-76fd272b-f5de-47a0-86c0-99acaa3e7438 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:14.792453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:14.793353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:14.794130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:14.798477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 166/321] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:14.800095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:14.829168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57de62-e3fc-44da-9895-57cf094a69b0 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.829500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57de62-e3fc-44da-9895-57cf094a69b0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.830001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57de62-e3fc-44da-9895-57cf094a69b0 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.831399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 167/322] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:14.836746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e895c-4ea3-4315-bc16-d0ce8717c845 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.837096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e895c-4ea3-4315-bc16-d0ce8717c845 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.837379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e895c-4ea3-4315-bc16-d0ce8717c845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.855862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-897e895c-4ea3-4315-bc16-d0ce8717c845 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:14.857075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-897e895c-4ea3-4315-bc16-d0ce8717c845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:14.859792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:14.904999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f300280-d6dc-41d0-b926-a3ab19671789 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:14.928657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 168/323] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:14.930389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09738-4641-4572-9912-23cfee0f1902 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.930677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09738-4641-4572-9912-23cfee0f1902 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.930948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb09738-4641-4572-9912-23cfee0f1902 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.932385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 169/324] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:14.937720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.938008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.938244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.939697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:14.940286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:14.940782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:14.940881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:14.941555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:14.941866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:14.943424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:14.958153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 156/325] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:14.958153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:14.959208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:14.960468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:15.030997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 170/326] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:15.040440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.068724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb20061-11a5-4d43-90f7-308881f3551c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.086690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:15.100199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:15.103732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.104414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.104687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.104952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.117006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:15.122759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:15.123840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:15.124686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 157/327] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:15.125715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.152530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe42ae-3aae-48a1-83ab-726bf4da1466 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.189948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.190891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.191166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.191443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] PATH_INFO: `/v3/users/04873ce505ca4ccab13c6bc6e4e32012` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.192775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=04873ce505ca4ccab13c6bc6e4e32012)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.194240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.200715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 171/328] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:15 2022] GET /identity/v3/users/04873ce505ca4ccab13c6bc6e4e32012 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:15.200715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.241572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254e561-f583-4427-ac2d-34a99892c3fe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.279345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.280028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.280331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.280597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.281868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.283487 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.285650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 158/329] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:15.286455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.307786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6389e3-92ec-4f9e-b31f-f9a1b0836fe4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.334763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:15.341914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:15.349648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.350283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.350552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.350854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.351562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:15.351912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.353306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.362264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 172/330] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:15.363107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.409168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ef6da-7eed-4253-83b7-8bdd04c6ed71 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.442695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.443318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.443598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.443939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.444714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] RBAC: Adding query filter params (role.id=806f65f717d34da3aa2fba83559fdc34, user.id=04873ce505ca4ccab13c6bc6e4e32012, scope.system=all) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:15.445018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.446454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.459546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 159/331] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 16:25:15 2022] GET /identity/v3/role_assignments?user.id=04873ce505ca4ccab13c6bc6e4e32012&scope.system=all&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:15.460401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.469899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375692b-87de-4239-8b0c-a093edd23361 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.508820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.509457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.509732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.509999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] PATH_INFO: `/v3/groups` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.510673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.512242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.534102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 173/332] 10.222.0.45 () {60 vars in 1134 bytes} [Mon Aug 1 16:25:15 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:15.596597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992b8a7-8a2b-4e81-bcb7-cf8537e7ef0e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.599728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:15.640277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72128287-a215-4557-bbd3-57faabed6010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.641214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72128287-a215-4557-bbd3-57faabed6010 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.641632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72128287-a215-4557-bbd3-57faabed6010 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.641904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72128287-a215-4557-bbd3-57faabed6010 None admin] PATH_INFO: `/v3/users/887844f9dad74652af24236883a6a757` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.643441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72128287-a215-4557-bbd3-57faabed6010 None admin] RBAC: Authorizing `identity:get_user(user_id=887844f9dad74652af24236883a6a757)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.644952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72128287-a215-4557-bbd3-57faabed6010 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.646618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 160/333] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:15 2022] GET /identity/v3/users/887844f9dad74652af24236883a6a757 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:15.652692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:15.708828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:15.769875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e7972-c8c8-4de1-a70a-890ef1701903 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:15.805945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb2db4-526e-4775-8a76-52b2072c0362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:15.806553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb2db4-526e-4775-8a76-52b2072c0362 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:15.806834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb2db4-526e-4775-8a76-52b2072c0362 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:15.807110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb2db4-526e-4775-8a76-52b2072c0362 None admin] PATH_INFO: `/v3/system/users/887844f9dad74652af24236883a6a757/roles/df635e925ad643e29aacbd1a1a32e0a2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:15.808620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb2db4-526e-4775-8a76-52b2072c0362 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=887844f9dad74652af24236883a6a757, role_id=df635e925ad643e29aacbd1a1a32e0a2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:15.810066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb2db4-526e-4775-8a76-52b2072c0362 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:15.820544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 174/334] 10.222.0.45 () {58 vars in 1334 bytes} [Mon Aug 1 16:25:15 2022] PUT /identity/v3/system/users/887844f9dad74652af24236883a6a757/roles/df635e925ad643e29aacbd1a1a32e0a2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:17.782934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4f6a-b83c-4cd9-8fb2-3b9d406f542d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:17.783539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4f6a-b83c-4cd9-8fb2-3b9d406f542d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:17.783708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4f6a-b83c-4cd9-8fb2-3b9d406f542d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:17.785478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 161/335] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:17.791311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc4faf-3660-46a8-8106-544d4f5822cf None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:17.791828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc4faf-3660-46a8-8106-544d4f5822cf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:17.791910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc4faf-3660-46a8-8106-544d4f5822cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:17.820834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e5bc4faf-3660-46a8-8106-544d4f5822cf None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:17.822039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e5bc4faf-3660-46a8-8106-544d4f5822cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:17.910860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 175/336] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:17.953315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee4f6a-b83c-4cd9-8fb2-3b9d406f542d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:18.035503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772807e0-eef6-43bd-897b-59650ed99fcf None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.035881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772807e0-eef6-43bd-897b-59650ed99fcf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.036078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772807e0-eef6-43bd-897b-59650ed99fcf None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.039080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 176/337] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:18.042944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.043220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.043491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.072617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:18.073848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:18.108555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee4f6a-b83c-4cd9-8fb2-3b9d406f542d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:18.126056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:18.179457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 177/338] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:18.189021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:18.189978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.190286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.190552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.210011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:18.215571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:18.216495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:18.217276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 162/339] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:18.222994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:18.223124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:18.289090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:18.408283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89b7f8-f757-442f-b025-201013fbb2a7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:18.437154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa121c3e-ea35-414f-9dde-e410e067ee8c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:18.450402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:18.451039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.451315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.451586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.452538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:18.452843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:18.454259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc48ae6-6fa3-4188-9eb1-f6b54b535549 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:18.475957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 178/340] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:18.477292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d5989-5337-4335-a48d-753cc1203484 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.477662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d5989-5337-4335-a48d-753cc1203484 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.477923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d5989-5337-4335-a48d-753cc1203484 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.479534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 179/341] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:18.481551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d5989-5337-4335-a48d-753cc1203484 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:18.524485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:18.524681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a0d8b-cc5b-4dd6-adcf-433fe2a0cb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:18.525207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a0d8b-cc5b-4dd6-adcf-433fe2a0cb6c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.525483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a0d8b-cc5b-4dd6-adcf-433fe2a0cb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.526056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a0d8b-cc5b-4dd6-adcf-433fe2a0cb6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.557426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a0d8b-cc5b-4dd6-adcf-433fe2a0cb6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-de3a0d8b-cc5b-4dd6-adcf-433fe2a0cb6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:18.562931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:18.563848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:18.564802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 163/342] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:18.565830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.566456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.566456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.615864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:18.617100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:18.625372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d5989-5337-4335-a48d-753cc1203484 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:18.696208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:18.697158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.697637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.697928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.764751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 164/343] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:18.765612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:18.808484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:18.831473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:18.832901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:18.872373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:18.906899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b4008-21fd-42c6-9725-a08101ba8c1c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:18.949249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:18.950021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.950258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.950531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.951265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:18.951567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:18.953659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322341b-3431-4b69-9fdc-2713db7a5ddd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:18.962961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 180/344] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:18.963854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:18.965674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 165/345] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:18.966936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19995d0-0f33-4a18-83dc-ccf65676db0c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:18.967219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19995d0-0f33-4a18-83dc-ccf65676db0c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:18.967493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19995d0-0f33-4a18-83dc-ccf65676db0c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:18.968741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 166/346] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:18.969424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19995d0-0f33-4a18-83dc-ccf65676db0c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.102669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b0a94-13dc-4334-91f7-ecd7f3468b75 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.102873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19995d0-0f33-4a18-83dc-ccf65676db0c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.108095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:19.143047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:19.150961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.151159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bd3c-3dd6-4988-aff3-a23b4b91019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.151605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.151824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bd3c-3dd6-4988-aff3-a23b4b91019b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.152002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.152101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bd3c-3dd6-4988-aff3-a23b4b91019b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.152264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.152361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bd3c-3dd6-4988-aff3-a23b4b91019b None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.153674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bd3c-3dd6-4988-aff3-a23b4b91019b None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:19.155694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bd3c-3dd6-4988-aff3-a23b4b91019b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:19.157436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 167/347] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:18 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:19.159290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.159803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.160063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.166176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:19.172488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:19.173584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:19.174874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 181/348] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:19.175639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.186481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:19.187683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:19.274364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 168/349] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:19.275242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.325998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eedc6-f50b-41e2-90c4-640ebe3eedfb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.369218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.369789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.370065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.370338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] PATH_INFO: `/v3/users/887844f9dad74652af24236883a6a757` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.371702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] RBAC: Authorizing `identity:get_user(user_id=887844f9dad74652af24236883a6a757)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:19.373199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:19.375194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 182/350] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/users/887844f9dad74652af24236883a6a757 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:19.375972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.390896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:19.419496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f7a2-256b-4a58-8b4f-d0654e654b7b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.456133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:19.478213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.479262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.479753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.480253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.487856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:19.489672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:19.491726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 169/351] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:19.492760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.507746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba8dba-dada-4f98-8762-643caac4def1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.544417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.545163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.545294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.545556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.546281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:19.546731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:19.548106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:19.558898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 183/352] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:19.563786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.613943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1806f60-a199-4a1b-9f08-d9c8e99c7407 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.653978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.654972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.655461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.656847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.672662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f941e4-9bd5-4e47-a90e-bc5e55fa2ac7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.675256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:19.685418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:19.686317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 170/353] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:19.687108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.698815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:19.703651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:19.716208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.717143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.717587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.718050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.719019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] RBAC: Adding query filter params (role.id=df635e925ad643e29aacbd1a1a32e0a2, user.id=887844f9dad74652af24236883a6a757, scope.system=all) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:19.719559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:19.721439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:19.735051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 184/354] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/role_assignments?user.id=887844f9dad74652af24236883a6a757&scope.system=all&role.id=df635e925ad643e29aacbd1a1a32e0a2 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:19.736342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:19.853928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2dcd7-c49f-4a56-8a07-410fafec0e6f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.855573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367744a-18a7-4825-9716-b04cd8840cf4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:19.902543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.903495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.904053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.904594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.912955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:19.914681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:19.914681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:19.914681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:19.915074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:19.915385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:19.917883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:19.929075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 171/355] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:19.930846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.005410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.043498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.045162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:20.046857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:20.090363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1235308-9dbc-4e03-8a8a-837f8777096e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.130029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:20.130956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.131398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.131672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.132442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:20.132747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:20.134161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0183c6f-b726-4bb9-9447-2a959cfdb180 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:20.148357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 172/356] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:20.149703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79be05-fddd-477c-8c3c-21d6fd4a509d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.149991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79be05-fddd-477c-8c3c-21d6fd4a509d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.150264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79be05-fddd-477c-8c3c-21d6fd4a509d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.151579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 173/357] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:20.157155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab186-71f4-4706-896d-0b6a5122e73a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.157412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab186-71f4-4706-896d-0b6a5122e73a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.157635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab186-71f4-4706-896d-0b6a5122e73a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.159431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 174/358] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:20.160401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a2b61-f689-4b3f-9218-493facf192c6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.160674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a2b61-f689-4b3f-9218-493facf192c6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.160947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a2b61-f689-4b3f-9218-493facf192c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.176190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 185/359] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:20.177180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.182242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a19a2b61-f689-4b3f-9218-493facf192c6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:20.183430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a19a2b61-f689-4b3f-9218-493facf192c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:20.270219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 175/360] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:20.272381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.273135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.273546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.292539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.301662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:20.303112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:20.316840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b00b72-c4ef-431b-a6f7-337169467487 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.362145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:20.363396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.363857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.364271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.377684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.393452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 176/361] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:20.397343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.451340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:20.452768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:20.528453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992e871-b0e9-4e47-b3ef-33daf5813f0c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.562882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.575225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:20.575917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.576193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.576460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.577728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:20.579195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:20.581088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 177/362] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:20.581777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.591781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 186/363] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:20.593152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.655376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.717841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5086ffc-8ff7-4f7b-b22d-7ec30b6bc244 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.722836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395795fb-c08f-4797-8b0b-c6cfa9db192f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.761936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:20.762997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.763377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.763744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.777861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:20.778519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.778794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.779060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.780824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:20.786945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:20.787938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 178/364] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:20.788726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.799840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:20.805405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:20.806528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:20.807499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:20.808696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 187/365] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:20.810558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:20.875702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.930999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ca48-b971-4780-9985-07ee73b982e5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.958992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:20.965577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a38db-309a-4c60-a42c-9a11ebc3c8af None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:20.980610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:20.981179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:20.981456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:20.981721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:20.982915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:20.984410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:20.986381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 179/366] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:20.986985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.014946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.014946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.014946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.014946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.016232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.018633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.021202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 188/367] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:21.022533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.118626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3e5c6-7d77-450b-bc80-c39e7483a25e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.165571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.165914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.166180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.166412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced63866-a200-4a15-9195-bc61150c1fae None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.166513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.167207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:21.167485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.170358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.176140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:21.183973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 180/368] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:21.183973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.210404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.211074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.211354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.211622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.212450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:21.212753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.214136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.225333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 189/369] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:21.226187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.263210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:21.321511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cf53d-5a78-4181-b063-f0b2f2ef4621 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.363851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e659b7-2e67-4d18-a46b-74373c20a759 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.366768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.368440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.369263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.369263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.370090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:21.370534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.372105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.380378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 181/370] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:21.381266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.424091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.425999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.426279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.426556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.427879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.429390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.432148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 190/371] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:21.433566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.483450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:21.528975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4f33-d7de-420a-96b9-4150b95e6687 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.548146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:21.569982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8003-2870-4e98-867e-f5cabddc45d7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.593513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.594178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.594448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.594723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.614338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:21.615838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:21.616568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:21.617118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:21.690275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.691901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.714358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:21.718490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:21.764250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:21.816730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 182/372] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:21.823293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.847856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:21.858748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 191/373] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:21.859907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:21.969298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab9c95-b35e-47de-9441-9736a0eb71f0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:21.980543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dee26-cf6d-4728-a4a0-2a25c5c8bf38 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.017986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.018889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.019361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.019846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.020819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=d8d875a7272446878db318488c144567) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:22.021304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.023063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.026177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.027040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.027451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.027916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.029232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.029757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 183/374] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=d8d875a7272446878db318488c144567&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:22.031002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.032816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.034385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 192/375] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:21 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:22.041164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.047708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:22.209921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c429bf-46f0-4821-aea3-3ff4a298795d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.214160 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:22.249512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa4b57-6109-49f9-9fd0-17405bb398fe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.258724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d146-277e-480a-b692-e2b729f25443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.259484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d146-277e-480a-b692-e2b729f25443 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.259886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d146-277e-480a-b692-e2b729f25443 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.260251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d146-277e-480a-b692-e2b729f25443 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.261640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d146-277e-480a-b692-e2b729f25443 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.263331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d146-277e-480a-b692-e2b729f25443 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.265393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 193/376] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:22.266909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610d97d-2581-41f4-b180-674dfa875626 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.267214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610d97d-2581-41f4-b180-674dfa875626 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.269485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610d97d-2581-41f4-b180-674dfa875626 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.272992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 194/377] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:22.273921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610d97d-2581-41f4-b180-674dfa875626 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.288371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21656554-76e8-4041-96ab-fcabd3109c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.289018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21656554-76e8-4041-96ab-fcabd3109c04 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.289293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21656554-76e8-4041-96ab-fcabd3109c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.289612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21656554-76e8-4041-96ab-fcabd3109c04 None admin] PATH_INFO: `/v3/projects/0c8a8aa32f9a4584972956058f337380` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.290778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21656554-76e8-4041-96ab-fcabd3109c04 None admin] RBAC: Authorizing `identity:get_project(project_id=0c8a8aa32f9a4584972956058f337380)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.292275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21656554-76e8-4041-96ab-fcabd3109c04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.293937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 184/378] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/projects/0c8a8aa32f9a4584972956058f337380 => generated 344 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:22.295077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.295417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.295688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.314695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:22.316006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:22.370372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:22.403901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610d97d-2581-41f4-b180-674dfa875626 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.412662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 185/379] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:22.420347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.462760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.463299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.463568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.463857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.464775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=d8d875a7272446878db318488c144567) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:22.465003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.466627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.472760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 195/380] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=d8d875a7272446878db318488c144567&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:22.473670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.539362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:22.575898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78fa2-eb89-4770-b3cc-9e17508098de None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.604705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399d38-e051-4a0c-877c-16d8ad594147 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.626339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.627833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.628350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.628817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] PATH_INFO: `/v3/groups/26562fbcb55446fbb0beb479a6671576` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.637085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] RBAC: Authorizing `identity:get_group(group_id=26562fbcb55446fbb0beb479a6671576)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.639913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.643395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 186/381] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/groups/26562fbcb55446fbb0beb479a6671576 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:22.648813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.652559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.653233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.653534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.655433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.665200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:22.692104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:22.700417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:22.701371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:22.702173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:22.703326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 196/382] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:22.711132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:22.890362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2aafa-e7c0-459e-a938-8597cd47f1c2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.938712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:22.940383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:22.941017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:22.941821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:22.943321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:22.944020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:22.946472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:22.962123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 197/383] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:22.962305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a165bc-6a46-491f-833b-04b752604353 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:22.965649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:23.010300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:23.012020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:23.012697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:23.013176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:23.014158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] RBAC: Adding query filter params (group.id=26562fbcb55446fbb0beb479a6671576, role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=0c8a8aa32f9a4584972956058f337380) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:23.014663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:23.016490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:23.023853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 187/384] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/role_assignments?group.id=26562fbcb55446fbb0beb479a6671576&scope.project.id=0c8a8aa32f9a4584972956058f337380&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 260 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:23.085032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:23.148842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6253a-fb9f-4cff-b14d-e29b74ea48b0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:23.198076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:23.198909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:23.199383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:23.199843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:23.205078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:23.207036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:23.209738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 198/385] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:22 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:23.216906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:23.223689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:23.357103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bf776-b259-4982-8ab7-68d94a7a4043 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:23.400637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee6120-0a1c-4737-a35f-12b2e3ecb9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:23.401437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee6120-0a1c-4737-a35f-12b2e3ecb9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:23.401794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee6120-0a1c-4737-a35f-12b2e3ecb9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:23.402163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee6120-0a1c-4737-a35f-12b2e3ecb9b2 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:23.403397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee6120-0a1c-4737-a35f-12b2e3ecb9b2 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:23.405466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee6120-0a1c-4737-a35f-12b2e3ecb9b2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:23.407511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 188/386] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:23 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:23.414698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:23.559372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c148f7-abbe-4c01-a2cd-9531bf4b6efd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:23.601711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:23.620646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d19ac7-6644-414d-8f3b-84f5f435b41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:23.621602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d19ac7-6644-414d-8f3b-84f5f435b41e None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:23.621962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d19ac7-6644-414d-8f3b-84f5f435b41e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:23.622326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d19ac7-6644-414d-8f3b-84f5f435b41e None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/groups/347b431a2b7a4381b43f1679ea474bff/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:23.625358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d19ac7-6644-414d-8f3b-84f5f435b41e None admin] RBAC: Authorizing `identity:create_grant(project_id=22cdac7ceb194d2c8b71fe379341ce04, group_id=347b431a2b7a4381b43f1679ea474bff, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:23.627510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d19ac7-6644-414d-8f3b-84f5f435b41e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:23.651021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 199/387] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 16:25:23 2022] PUT /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/groups/347b431a2b7a4381b43f1679ea474bff/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:24.068424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21db68-8d99-4dc9-8fc4-4e55d19c4233 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.068971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21db68-8d99-4dc9-8fc4-4e55d19c4233 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.069046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21db68-8d99-4dc9-8fc4-4e55d19c4233 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.071060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 189/388] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:24.077361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.077783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.078196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.109142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:24.111025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:24.213031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 200/389] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:24.257504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21db68-8d99-4dc9-8fc4-4e55d19c4233 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:24.365467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:24.420102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21db68-8d99-4dc9-8fc4-4e55d19c4233 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:24.472569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc5fd1-4ea1-4066-ad2f-fab866fafadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:24.473407 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5fd1-4ea1-4066-ad2f-fab866fafadf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.473739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5fd1-4ea1-4066-ad2f-fab866fafadf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.474077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc5fd1-4ea1-4066-ad2f-fab866fafadf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.490948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc5fd1-4ea1-4066-ad2f-fab866fafadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-68bc5fd1-4ea1-4066-ad2f-fab866fafadf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:24.496906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:24.497784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:24.498654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:24.501294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 190/390] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:24.510091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:24.625333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea269a6-9730-46fb-afb4-947bd6aabf95 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.626043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea269a6-9730-46fb-afb4-947bd6aabf95 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.626630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea269a6-9730-46fb-afb4-947bd6aabf95 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.630562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 191/391] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:24.635621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.635742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045db88-edb3-4ae4-81f3-19684763ba97 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:24.636250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.636731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.652890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:24.679644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:24.682456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:24.698091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:24.698722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.698995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.699265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.700048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:24.700353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:24.702020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:24.717735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 201/392] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:24.724200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:24.815435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 192/393] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:24.855889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:24.919940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb067835-33d0-44e5-bb29-3d4911412cff None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:24.950795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:24.962517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ef195-2ab9-450a-b426-d64f7f37bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:24.963160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef195-2ab9-450a-b426-d64f7f37bb64 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.963437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef195-2ab9-450a-b426-d64f7f37bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.963707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef195-2ab9-450a-b426-d64f7f37bb64 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.964897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ef195-2ab9-450a-b426-d64f7f37bb64 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:24.966657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ef195-2ab9-450a-b426-d64f7f37bb64 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:24.968701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 202/394] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:24 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:24.969954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff4537-8a6b-460f-a61f-a6dabcbaa3e8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:24.970556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff4537-8a6b-460f-a61f-a6dabcbaa3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:24.970987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff4537-8a6b-460f-a61f-a6dabcbaa3e8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:24.972280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 203/395] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:24.980940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff4537-8a6b-460f-a61f-a6dabcbaa3e8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:24.996278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635b46e-2a6c-44e9-ad20-0b46f701b8b1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.030576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd009c-2128-4e36-8c17-2b6bcbf3f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.031219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd009c-2128-4e36-8c17-2b6bcbf3f164 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.031491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd009c-2128-4e36-8c17-2b6bcbf3f164 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.031785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd009c-2128-4e36-8c17-2b6bcbf3f164 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.034674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:25.042434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd009c-2128-4e36-8c17-2b6bcbf3f164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-57fd009c-2128-4e36-8c17-2b6bcbf3f164 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:25.047645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:25.048770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:25.049557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 193/396] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:25.054201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.054201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.054201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.069013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:25.070199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:25.097789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff4537-8a6b-460f-a61f-a6dabcbaa3e8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.137499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.143784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.143920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 194/397] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:25.143920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.144554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.145016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.147278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.149047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.151397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 204/398] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:24 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:25.157270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.192623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:25.241357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1ba2b-139a-47f5-be22-6482441df4c3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.276100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.276724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.276996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.277259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.278110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:25.278261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.279639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.290073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 195/399] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:25.290889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.293381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357963f-ce54-4edc-9603-f7b952ffafaa None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.311283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:25.335646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.336731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.337149 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.337556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/groups/347b431a2b7a4381b43f1679ea474bff/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.340316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=22cdac7ceb194d2c8b71fe379341ce04, group_id=347b431a2b7a4381b43f1679ea474bff, role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.341921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.364041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 205/400] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 16:25:25 2022] PUT /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04/groups/347b431a2b7a4381b43f1679ea474bff/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:25:25.365291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.395969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b689f-f07a-4651-9b82-8916d9267b0c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.411656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:25.433335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.433966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.434252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.434508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.445999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:25.451212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:25.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:25.457579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 196/401] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:25.460758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.594479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c1c7c-1b65-4a81-854f-3f7e6da107f0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.635621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.636342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.636755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.637033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.638289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.639920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.642484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 206/402] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:25.645552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba50058-f18d-485a-b7fb-07e413d2f16d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.647852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.685491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.686194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.686542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.686871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.687661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:25.688049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.689956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.699815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 197/403] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:25.703611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:25.708630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.772644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a9846-3139-46b1-8efa-be3ad5b828e0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.793672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:25.813861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.815008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.815426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.815884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.817331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.819186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.821535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 207/404] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:25.826909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.892284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e360a1-82e3-4f21-8918-939dab33f991 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.933554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e9fd2-5b11-4f32-af17-8b6dab334ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:25.934089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9fd2-5b11-4f32-af17-8b6dab334ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.934369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9fd2-5b11-4f32-af17-8b6dab334ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.934645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e9fd2-5b11-4f32-af17-8b6dab334ce8 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.935827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e9fd2-5b11-4f32-af17-8b6dab334ce8 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:25.937288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e9fd2-5b11-4f32-af17-8b6dab334ce8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:25.943771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 198/405] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:25.945065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6816fc-7efa-458c-a4e4-c058ec35b869 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:25.945705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6816fc-7efa-458c-a4e4-c058ec35b869 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:25.945705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6816fc-7efa-458c-a4e4-c058ec35b869 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:25.947177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 199/406] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:25.947818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6816fc-7efa-458c-a4e4-c058ec35b869 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:25.961855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879dd05-83ab-47d6-a45b-3fea34eae5ed None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:25.974807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:26.002179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9373d-5e43-427d-92cd-1095dd679d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.003508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9373d-5e43-427d-92cd-1095dd679d9a None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.004063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9373d-5e43-427d-92cd-1095dd679d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.004529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9373d-5e43-427d-92cd-1095dd679d9a None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567/groups/347b431a2b7a4381b43f1679ea474bff/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.006867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9373d-5e43-427d-92cd-1095dd679d9a None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d875a7272446878db318488c144567, group_id=347b431a2b7a4381b43f1679ea474bff, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:26.008641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9373d-5e43-427d-92cd-1095dd679d9a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:26.029567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 208/407] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 16:25:25 2022] PUT /identity/v3/projects/d8d875a7272446878db318488c144567/groups/347b431a2b7a4381b43f1679ea474bff/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:26.031194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.031798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.032263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.057168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:26.058804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:26.104535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6816fc-7efa-458c-a4e4-c058ec35b869 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:26.115643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:26.176092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.179490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.181039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.182778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.185897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:26.187820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:26.190505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 200/408] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:25 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:26.199600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:26.203528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 209/409] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:26.228550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:26.336477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e708e8a-3f42-405a-a6fb-ac7e251969b5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:26.348309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:26.351124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516a92-293d-4369-a046-e38e0f649f8e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:26.381452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed3a6-32fa-425b-b9cd-f234d19582b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.382477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed3a6-32fa-425b-b9cd-f234d19582b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.382891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed3a6-32fa-425b-b9cd-f234d19582b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.383297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed3a6-32fa-425b-b9cd-f234d19582b8 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567/groups/347b431a2b7a4381b43f1679ea474bff/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.385896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eed3a6-32fa-425b-b9cd-f234d19582b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8d875a7272446878db318488c144567, group_id=347b431a2b7a4381b43f1679ea474bff, role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:26.387718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eed3a6-32fa-425b-b9cd-f234d19582b8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:26.390376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.391131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.391518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.391885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.403931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:26.404696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 201/410] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 16:25:26 2022] PUT /identity/v3/projects/d8d875a7272446878db318488c144567/groups/347b431a2b7a4381b43f1679ea474bff/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:26.406226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0616fca-109b-450b-b999-a0bc896c6573 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.406729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0616fca-109b-450b-b999-a0bc896c6573 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.407181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0616fca-109b-450b-b999-a0bc896c6573 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.409600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 202/411] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:26.409936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:26.410983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:26.412034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 210/412] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:26.415943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.416786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.417249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.417668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:26.438540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:26.440631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:26.471875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:26.522117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 203/413] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:26.535650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d3fcc-4c23-4a31-a36f-dcb15d349abe None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:26.560651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:26.583065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.583702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.584242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.584516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.585254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:26.585555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:26.586992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:26.611927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 211/414] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:26.633882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:26.677101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:26.809238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2e3f3-fc5f-47ba-82b0-f12840e47bc6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:26.843684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b270ff-54d2-4c5f-95a4-8716a919f9bd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:26.890434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.892271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.894273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.894772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.918253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:26.919455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:26.920335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:26.922495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:26.922976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] PATH_INFO: `/v3/groups/26562fbcb55446fbb0beb479a6671576` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:26.927473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] RBAC: Authorizing `identity:get_group(group_id=26562fbcb55446fbb0beb479a6671576)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:26.929220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:26.932914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 212/415] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:26 2022] GET /identity/v3/groups/26562fbcb55446fbb0beb479a6671576 => generated 229 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:26.933406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:26.934255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:26.935577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:26.937275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:26.939932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 204/416] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:26.947271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:27.001034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:27.151874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:27.156589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593d844-6ee2-40e7-b3a9-1adf4d7b9c41 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:27.180719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712f04-7f98-4247-8a0d-5f6418af84f0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:27.232669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:27.233326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:27.233604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:27.233871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:27.234593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:27.234894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:27.236565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:27.255714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 205/417] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:27.257359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:27.259092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:27.259092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:27.259092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] PATH_INFO: `/v3/projects/0c8a8aa32f9a4584972956058f337380` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:27.259865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] RBAC: Authorizing `identity:get_project(project_id=0c8a8aa32f9a4584972956058f337380)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:27.261348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:27.263672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:27.264920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 213/418] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:26 2022] GET /identity/v3/projects/0c8a8aa32f9a4584972956058f337380 => generated 344 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:27.271224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:27.444227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:27.479273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bcb69-d0d8-40a9-907f-37f9be184c93 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:27.493237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c4601-62e6-4067-8510-3bbd6fd90156 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:27.558837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:27.559472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:27.559750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:27.560045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:27.567368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:27.568123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:27.568401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:27.568670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] PATH_INFO: `/v3/projects/0c8a8aa32f9a4584972956058f337380/groups/26562fbcb55446fbb0beb479a6671576/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:27.576988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] RBAC: Authorizing `identity:create_grant(project_id=0c8a8aa32f9a4584972956058f337380, group_id=26562fbcb55446fbb0beb479a6671576, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:27.578513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:27.600875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 214/419] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Aug 1 16:25:27 2022] PUT /identity/v3/projects/0c8a8aa32f9a4584972956058f337380/groups/26562fbcb55446fbb0beb479a6671576/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:27.656115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:27.657643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:27.753744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:27.772562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 206/420] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:25:27.780954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:27.811350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:27.916335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa0c42-1b82-4270-bb36-7ec18542571b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:27.957302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97b789-eafb-4e2e-bbdd-589595b956df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:27.957965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97b789-eafb-4e2e-bbdd-589595b956df None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:27.958239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97b789-eafb-4e2e-bbdd-589595b956df None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:27.958506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97b789-eafb-4e2e-bbdd-589595b956df None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:27.961679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97b789-eafb-4e2e-bbdd-589595b956df None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:27.963136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97b789-eafb-4e2e-bbdd-589595b956df None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:27.966000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 215/421] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:27 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:27.971569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:28.097756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee4c9f-7829-4c9c-ace4-9133df3fee23 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.098180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee4c9f-7829-4c9c-ace4-9133df3fee23 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.098303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee4c9f-7829-4c9c-ace4-9133df3fee23 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.099631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 216/422] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:28.108702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442bd4a-e113-4dc3-9f6a-352dde25f709 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:28.114721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.114991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.115266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.141232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:28.142670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:28.188043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:28.188782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.189194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.189461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.193535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:28.194997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:28.196681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 207/423] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:27 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:28.204939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:28.267846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 217/424] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:28.288759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:28.313617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:28.343383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:28.402550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bcbac-989f-4ec0-bef7-80db69602286 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:28.440441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:28.441264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.441542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.442135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.446770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fb0cf-daeb-45ae-bd23-344ecaad647c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:28.454298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:28.459982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:28.460868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:28.461677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:28.466064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 218/425] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:28.467734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:28.525755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:28.526384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.526720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.526988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.527848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:28.528153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:28.529586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:28.546331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 208/426] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:28 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 611 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:28.580225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:28.600268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3b6b8-1f3d-4719-a829-b6025c80771b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:28.637990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:28.638611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.638884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.639151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.639973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:28.640276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:28.641798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269cc8e-afdc-4f04-a35c-ba8bfd35ff1f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:28.656043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 219/427] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:28.657938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:28.686275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32f0f1-1772-4e4e-819b-afd7e5f94291 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.686580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32f0f1-1772-4e4e-819b-afd7e5f94291 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.687642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32f0f1-1772-4e4e-819b-afd7e5f94291 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.690484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 220/428] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:28.697296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.698119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.698119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.725064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:28.726418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:28.802344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 221/429] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:28.824519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8e4c7-b393-4c5b-b99f-3fce7d6a280b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:28.830963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:28.848509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:28.880189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b24f5-90f2-4834-8c70-11e55a5e0076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:28.881666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b24f5-90f2-4834-8c70-11e55a5e0076 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:28.882491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b24f5-90f2-4834-8c70-11e55a5e0076 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:28.882970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b24f5-90f2-4834-8c70-11e55a5e0076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:28.952084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.006915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396fa7f-b17d-424f-b2b9-16fd2f1e1e4e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.009922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b24f5-90f2-4834-8c70-11e55a5e0076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.011863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b24f5-90f2-4834-8c70-11e55a5e0076 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.049047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8b023-91e6-4a0b-8ac8-ea5f9605d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.050114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8b023-91e6-4a0b-8ac8-ea5f9605d040 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.050385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8b023-91e6-4a0b-8ac8-ea5f9605d040 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.050652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8b023-91e6-4a0b-8ac8-ea5f9605d040 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.064670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8b023-91e6-4a0b-8ac8-ea5f9605d040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2fb8b023-91e6-4a0b-8ac8-ea5f9605d040 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:29.071034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:29.072011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:25:29.072800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:29.073446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 222/430] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:29.075790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2fe8e-28f4-4f02-a9de-b52ad593be48 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.075790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2fe8e-28f4-4f02-a9de-b52ad593be48 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.075790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2fe8e-28f4-4f02-a9de-b52ad593be48 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.076469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 223/431] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:29.080269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2fe8e-28f4-4f02-a9de-b52ad593be48 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.157043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 209/432] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:29.158686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.158973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.159247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.186578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:29.190828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:29.201678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.223710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2fe8e-28f4-4f02-a9de-b52ad593be48 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.254484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.264178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.264818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.265096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.265367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.266105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:29.266408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.267924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.272138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 210/433] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:29.273118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.288656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 224/434] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:29.303814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.388952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7bd3f-5305-468e-a6a5-61a411af942f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.399527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49704d02-195a-4015-9aa7-1ddb699102b7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.429701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.430640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.431144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.431422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] PATH_INFO: `/v3/projects/22cdac7ceb194d2c8b71fe379341ce04` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.432689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] RBAC: Authorizing `identity:get_project(project_id=22cdac7ceb194d2c8b71fe379341ce04)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.434141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.440793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 211/435] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/projects/22cdac7ceb194d2c8b71fe379341ce04 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:29.440971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.440971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.441152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.441249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.441249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.458576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.503585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.514970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.516509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.555062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25308-36a6-4957-8a81-2ca3342ae122 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.591308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.591981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.592259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.592528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.603216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:29.608168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:29.609039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:29.609819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 212/436] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:29.610465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.628582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 225/437] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:25:29.633276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.706498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.732697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.738064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aec1e-5982-4817-8df5-972248f9d7d0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.744111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e519220-ba29-4f13-80ea-b042d03cb65d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.776084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.776732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.777004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.777270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.778410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.779887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.780653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.781335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.781611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.781894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.782644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:29.782951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.784388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.786822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 213/438] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:29.787705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.793451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 226/439] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:29.794131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.903038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f194-3515-4beb-9164-98b212ec0749 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.905474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d10660-29cf-4bd2-8a55-13336ba69510 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:29.942822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.943459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:29.943752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.944060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.944166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:29.944380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:29.944617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.944720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:29.945463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=22cdac7ceb194d2c8b71fe379341ce04) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:29.945769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.945860 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:29.947205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.947308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:29.953418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 214/440] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:29.953418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.959101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 227/441] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=22cdac7ceb194d2c8b71fe379341ce04&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:29.960803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:29.968900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:29.990888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.088982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edf47c-7f5d-4954-802c-28870be59d6b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.094896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa90f7-8a21-4db2-bacd-daafab94e379 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.132188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b04df-6036-4f3a-8c3a-d098598931d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.132835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b04df-6036-4f3a-8c3a-d098598931d4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.133110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b04df-6036-4f3a-8c3a-d098598931d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.133379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b04df-6036-4f3a-8c3a-d098598931d4 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.133482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.134054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.134333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.134528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b04df-6036-4f3a-8c3a-d098598931d4 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.134629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.135988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b04df-6036-4f3a-8c3a-d098598931d4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.137765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 228/442] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:30.141153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91e75b-f138-42ef-b88c-912c88518fe7 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.141434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91e75b-f138-42ef-b88c-912c88518fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.141703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91e75b-f138-42ef-b88c-912c88518fe7 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.143609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 229/443] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:30.147598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91e75b-f138-42ef-b88c-912c88518fe7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.217059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.218564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.230261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.250433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.270677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91e75b-f138-42ef-b88c-912c88518fe7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.315031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.315662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.315968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.316238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.317034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=806f65f717d34da3aa2fba83559fdc34, scope.project.id=d8d875a7272446878db318488c144567) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:30.317345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.318789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a271d8-4624-4fbb-a9c2-3e8a74c7e3f5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.325033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 230/444] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=d8d875a7272446878db318488c144567&role.id=806f65f717d34da3aa2fba83559fdc34 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:30.326228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.326511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.326784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.345496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 215/445] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:30.346381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:30.347834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:30.351401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.428548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 231/446] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:30.452876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.463638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ead7c-e9b7-42d5-b264-f013d46f6954 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.476223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.506465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.507120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.507399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.507672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] PATH_INFO: `/v3/projects/d8d875a7272446878db318488c144567` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.508946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] RBAC: Authorizing `identity:get_project(project_id=d8d875a7272446878db318488c144567)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.510482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.513402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 216/447] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/projects/d8d875a7272446878db318488c144567 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:30.519292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.546612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.572958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf98ef-a2d4-4e46-9767-56a31dce812d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.613705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.614285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.614604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.614832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.627031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:30.632427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:30.633392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:30.634233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:30.635367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 232/448] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:30.638271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6efff2-0b93-4ddc-8de5-3f1883903123 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.641733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.688970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.690183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.690632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.691061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] PATH_INFO: `/v3/groups/347b431a2b7a4381b43f1679ea474bff` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.692557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] RBAC: Authorizing `identity:get_group(group_id=347b431a2b7a4381b43f1679ea474bff)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.694015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.695641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 217/449] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/groups/347b431a2b7a4381b43f1679ea474bff => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:30.701993 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.748368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.757642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9152-1e07-499e-8a8b-fa66180bfbd9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.791516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.792166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.792443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.792712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.793435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:30.793743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.795133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.800305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:30.803485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0200c9-1ea4-4df0-8003-4a6f6ab333c3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.808766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 233/450] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:30.809646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:30.839250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.839907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.840187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.840460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:30.841249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] RBAC: Adding query filter params (group.id=347b431a2b7a4381b43f1679ea474bff, role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=d8d875a7272446878db318488c144567) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:30.841558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:30.843000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:30.849388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 218/451] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/role_assignments?group.id=347b431a2b7a4381b43f1679ea474bff&scope.project.id=d8d875a7272446878db318488c144567&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:30.918146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f2d65-6fcc-49a6-8090-2ffa26bcca70 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:30.954907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:30.955565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:30.955871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:30.956141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:31.030261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:31.034242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:31.035700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:31.149346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 234/452] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:31.156171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:31.266287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2fc8a-10ab-4d4b-ba92-f6e0617e6f4d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:31.277885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:31.305546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:31.306385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:31.306675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:31.306956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] PATH_INFO: `/v3/projects/0c8a8aa32f9a4584972956058f337380` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:31.308271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] RBAC: Authorizing `identity:get_project(project_id=0c8a8aa32f9a4584972956058f337380)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:31.309735 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:31.311745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 219/453] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:25:31 2022] GET /identity/v3/projects/0c8a8aa32f9a4584972956058f337380 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:31.317181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:31.439754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:31.443388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb85a14-9440-4539-8f4c-f722696c5cd3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:31.484834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cb8fa-4a2c-4b9d-a0fe-2805d93521aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:31.485476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cb8fa-4a2c-4b9d-a0fe-2805d93521aa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:31.485830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cb8fa-4a2c-4b9d-a0fe-2805d93521aa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:31.486095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cb8fa-4a2c-4b9d-a0fe-2805d93521aa None admin] PATH_INFO: `/v3/groups/26562fbcb55446fbb0beb479a6671576` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:31.487448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cb8fa-4a2c-4b9d-a0fe-2805d93521aa None admin] RBAC: Authorizing `identity:get_group(group_id=26562fbcb55446fbb0beb479a6671576)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:31.488934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cb8fa-4a2c-4b9d-a0fe-2805d93521aa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:31.490551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 235/454] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Aug 1 16:25:31 2022] GET /identity/v3/groups/26562fbcb55446fbb0beb479a6671576 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:31.500576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:31.624219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6def37-7500-4040-b787-65ea9ca3ead4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:31.666966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:31.667667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:31.667978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:31.668248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:31.669039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] RBAC: Adding query filter params (group.id=26562fbcb55446fbb0beb479a6671576, role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=0c8a8aa32f9a4584972956058f337380) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:31.669350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:31.670824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306216a-8ecd-4cdb-924e-1193a7d1d100 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:31.678014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 220/455] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Aug 1 16:25:31 2022] GET /identity/v3/role_assignments?group.id=26562fbcb55446fbb0beb479a6671576&scope.project.id=0c8a8aa32f9a4584972956058f337380&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:34.156016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc3bc6-3876-4a50-ba0b-aabe3310b2b1 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.157073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc3bc6-3876-4a50-ba0b-aabe3310b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.157825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc3bc6-3876-4a50-ba0b-aabe3310b2b1 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.159918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 236/456] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:34.165067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.165887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.166350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.193817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:34.195034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:34.237034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba15ad-ccd9-43d0-b852-5750d46f21d2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.237651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba15ad-ccd9-43d0-b852-5750d46f21d2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.238107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba15ad-ccd9-43d0-b852-5750d46f21d2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.241038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 237/457] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:34.245218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.245755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.246223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.252076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:34.276909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:34.279122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:34.326594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 221/458] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:34.376658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:34.387498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 238/459] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:34.433354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:34.516897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0aa6-a752-4bf9-9c3e-a32273bc0d27 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:34.556137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:34.556798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.557080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.557355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.569313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:34.570970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:34.588731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:34.589703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:34.590565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:34.590565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:34.590565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:34.590565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:34.590565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 222/460] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:34.597254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:34.623028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:34.629727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff314a0-3b9f-4342-b15d-0d09a45c9c55 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:34.649805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:34.714929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:34.715998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.716510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.716991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.746800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:34.748700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d4713a4a-3736-4e01-8ece-1e15fc0d2169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:34.766426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:34.768246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:34.769091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:34.769091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:34.769091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:34.769091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:34.771386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 239/461] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:34.773052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee79eb2-f87f-4c70-bfc4-730ff3484a26 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.773596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee79eb2-f87f-4c70-bfc4-730ff3484a26 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.774086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee79eb2-f87f-4c70-bfc4-730ff3484a26 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.775788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 240/462] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:34.776957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee79eb2-f87f-4c70-bfc4-730ff3484a26 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:34.794593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804b85d-9002-4222-8836-b29deb5b95e0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:34.834409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:34.835580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.836091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.836683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.837482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:34.837856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:34.839417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40178-e468-4013-a43a-d5d077d2b9a5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:34.858023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 223/463] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:34.858023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.858023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.858023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.875187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:34.876568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:34.905792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee79eb2-f87f-4c70-bfc4-730ff3484a26 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:34.947340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:34.947988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.948267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.948546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.949238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:34.949541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:34.956072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 224/464] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:34.956239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4646179-4098-41b4-b4ca-f3ee311f4871 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:34.956325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:34.973402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 241/465] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:34.974736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f50d08-648c-4069-a9e6-eb035d1e79f2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.975021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f50d08-648c-4069-a9e6-eb035d1e79f2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.975292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f50d08-648c-4069-a9e6-eb035d1e79f2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:34.976634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 242/466] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:34.979334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:34.982870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:34.983278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:34.983555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.002965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:35.004205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:35.007678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.068796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e802c-6a7a-4bc6-9b06-bb39b5d89557 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.077331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 243/467] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:35.078616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.114900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.115789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.116009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.116148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.117122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.118607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.147803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 225/468] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:34 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:35.147803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.216937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf672-fd75-4a74-9753-fb455571a48b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.258803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.259444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.259924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.260301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.261218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.262750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.264419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.279949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.298444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 244/469] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:35 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:35.298444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.330371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24b880-a8e8-4b6e-8ef5-45785b9cdcb5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.379067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.379884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.380165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.380436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.394858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.396387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:35.414246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:35.415162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:35.416464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:35.416464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:35.416464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:35.416464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:35.419922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 226/470] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:35.421881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.442874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515ff89-e076-44aa-89ed-96eac584fffd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.485513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.486174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.486455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.486791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.499533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.501239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:35.517089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:35.519213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:35.521628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:35.521628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:35.521628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:35.521628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:35.521628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 245/471] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:35.525379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.545033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61eacc-3555-4ae6-836e-f89dd9f52fc0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.579649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.597402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.597870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.598988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.598988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.598988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.599609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:35.600011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.601500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.616307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 227/472] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:35.623527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.669776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f7d3f-0edf-4a4a-8189-c4387e2c58f4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.730343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.730968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.731245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.731646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.732415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:35.732730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.734400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8f331-dd9e-4312-a87f-d9a99308da3a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.752055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefcef7-bbf8-46ef-be98-389252e2a3e0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.753541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 246/473] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:35.758333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6c192-7d84-4b4e-a99c-b00d5c6085e1 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.758333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6c192-7d84-4b4e-a99c-b00d5c6085e1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.758333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6c192-7d84-4b4e-a99c-b00d5c6085e1 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.758333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 247/474] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:35.759252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6c192-7d84-4b4e-a99c-b00d5c6085e1 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:35.802108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88b5e-a03f-4714-b550-a844da0e1ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.802763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88b5e-a03f-4714-b550-a844da0e1ec5 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.803037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88b5e-a03f-4714-b550-a844da0e1ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.803488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88b5e-a03f-4714-b550-a844da0e1ec5 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.805136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88b5e-a03f-4714-b550-a844da0e1ec5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.806783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88b5e-a03f-4714-b550-a844da0e1ec5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:35.836747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 228/475] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:35 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:25:35.838356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.838643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.839351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.863187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:35.865063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:35.904448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6c192-7d84-4b4e-a99c-b00d5c6085e1 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:35.923960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.932081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:35.989240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:35.990287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:35.990789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:35.991275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:35.992217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:35.992561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 229/476] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:35.994121 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:36.016785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:36.042825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 248/477] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:35 2022] POST /identity/v3/users => generated 249 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:36.152759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e823-1e2f-46fa-933c-2f963a9a8093 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:36.193876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:36.194574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:36.194855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:36.195122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:36.195900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:36.197392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:36.208137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.sql.core [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 16:25:36.208137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 16:25:36.208137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [parameters: {'id': 'd02830bf76ac46adabec149fa0d2710f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 16:25:36.208137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80348) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 16:25:36.225592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:25:36.226528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application yield resource Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise self.value Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application session.commit() Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 16:25:36.227421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.flush() Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application compat.raise_( Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 16:25:36.228307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:25:36.229208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:25:36.230018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [parameters: {'id': 'd02830bf76ac46adabec149fa0d2710f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:36.230774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 16:25:36.231617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:36.236966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 230/478] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 16:25:36 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:36.237101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:36.348748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8823066-10dd-4367-b8c2-9098e376b7d5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:36.384238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320dfc3-450a-4e35-b95e-cc210c4a446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:36.384889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320dfc3-450a-4e35-b95e-cc210c4a446a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:36.385164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320dfc3-450a-4e35-b95e-cc210c4a446a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:36.385434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320dfc3-450a-4e35-b95e-cc210c4a446a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:36.396706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320dfc3-450a-4e35-b95e-cc210c4a446a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5320dfc3-450a-4e35-b95e-cc210c4a446a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:36.402200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:36.403134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:25:36.404001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:36.407872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 249/479] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:25:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:36.409647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:36.445471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:36.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf5603-df55-47b9-9a72-265f7b74ea4d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:36.576299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:36.576994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:36.577275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:36.577539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:36.578277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:36.578585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:36.580159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bab891-7f7e-439f-b760-a6277470f407 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:36.591987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 231/480] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:37.863240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e425a1-00a4-42c3-8682-6b3bee5b12b2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:37.863841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e425a1-00a4-42c3-8682-6b3bee5b12b2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:37.863922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e425a1-00a4-42c3-8682-6b3bee5b12b2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:37.865370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 250/481] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:37.870762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:37.871156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:37.871331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:37.880274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece89e0a-10df-43eb-b5e2-bb5dd928a3f2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:37.880621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece89e0a-10df-43eb-b5e2-bb5dd928a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:37.880963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece89e0a-10df-43eb-b5e2-bb5dd928a3f2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:37.882767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 251/482] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:37.893234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:37.893524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:37.893854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:37.912887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:37.914113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:37.925232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:37.926796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:37.973774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:37.987321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 232/483] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:38.024552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.037128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 252/484] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:38.067806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:38.106096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.208922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de4167-6b47-4f86-80a8-3a5151a1a19f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.303474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea4e07-1fa0-4854-a513-e6e802cd76d3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.303957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.304685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.304969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.305240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.335323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:38.340367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:38.341232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:38.342028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:38.346548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.347099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 233/485] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:38.347282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.347465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.347738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.353187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.360658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:38.371650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:38.373827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:38.376134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 253/486] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:38.380997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.401862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:38.515100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503652f8-05ae-4dc3-9102-3cedc9ad23e8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.539825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:38.560482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be46f3-5bb5-420d-a7dd-1f7b45285c6f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.561192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.561814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.562083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.562509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.563249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:38.563727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:38.565547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdda465-1722-48a4-8145-41c52327f62a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:38.575298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 254/487] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:38.576596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d94d-a90c-4f93-95ed-971239bee004 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.576944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d94d-a90c-4f93-95ed-971239bee004 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.577221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d94d-a90c-4f93-95ed-971239bee004 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.578433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 255/488] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:38.583556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d94d-a90c-4f93-95ed-971239bee004 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.608852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.609610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.609942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.610268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.611275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:38.611655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:38.613185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e41806-4683-4cd0-b841-d04e406159d9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:38.623606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 234/489] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:38.625401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.625748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.626161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.648525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:38.649897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:38.672488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:38.696758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d94d-a90c-4f93-95ed-971239bee004 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.734335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 235/490] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:38.735424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.744915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.745687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.746004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.746282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.772773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:38.775388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2dd8e2f9-983b-4a75-8682-9a851db5cef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:38.793824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:38.794834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:38.796240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:38.796240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:38.796240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:38.796240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:38.796240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 256/491] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:38.797282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923275c-6e25-4c55-be97-6826e1808e16 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.797586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923275c-6e25-4c55-be97-6826e1808e16 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.797959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923275c-6e25-4c55-be97-6826e1808e16 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.799794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 257/492] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:38.801511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923275c-6e25-4c55-be97-6826e1808e16 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.829492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:38.855096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72632-3fea-4add-b37e-c3c814fd0ff2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.891241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.892510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.892790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.893060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.903874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:38.905760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:38.921994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923275c-6e25-4c55-be97-6826e1808e16 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:38.922489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:38.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:38.925214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:38.925214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:38.925214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:38.925214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:38.925214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 236/493] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:38.925214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:38.955846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:38.966029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806c408-9925-4a72-9571-4e8a85ec70d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:38.966688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806c408-9925-4a72-9571-4e8a85ec70d9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.966967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806c408-9925-4a72-9571-4e8a85ec70d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.967238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806c408-9925-4a72-9571-4e8a85ec70d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:38.980333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806c408-9925-4a72-9571-4e8a85ec70d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f806c408-9925-4a72-9571-4e8a85ec70d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:38.991719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:38.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:38.993884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:38.996443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 258/494] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:38.998321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:38.998825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:38.999631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.020311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:39.021525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:39.056509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec59b4-1c68-4601-992e-5da18fb2a6e3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.089354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.098193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.098818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.099316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 259/495] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:39.099902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.100134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.100607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.101486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:39.102004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.103584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.123925 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 237/496] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:39.125219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.225293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15357bb4-93f0-4af4-a29d-1e581b30b6ab None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.232975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3d023-0344-4449-9f85-38ddb229b765 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.245208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.265129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.265782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.266041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.266304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.267198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:39.267727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.269197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.271743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.272398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.272669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.272934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.273643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:39.273939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.275344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.281041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 260/497] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:39.286614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.287918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 238/498] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:39.288918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.343462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.395877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a53be-098d-49c9-8715-0aa606996bba None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.400242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07401269-f32b-4158-a9a0-bcc31ad82a2e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.431082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.431731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.432045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.432313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.434688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.435325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.435598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.435891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.441250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.442713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.446671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:39.452019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:39.453006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:39.453777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 261/499] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:39.454645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:39.455796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:39.456649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:39.457504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 239/500] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:39.458049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.498877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.559113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de55f-ffc7-4196-bb6d-dc09c4c787c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.562004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1612c5-3fd1-46f3-8692-0ebb9b50ff99 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.572548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.594772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.595716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.596031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.596304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.597541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.598271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.598545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.598812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.604489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.605945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.610266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.612153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-65b43925-88e5-4567-8882-b6963bad98ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:39.619071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:39.620006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:39.621138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 262/501] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:39.621903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ee0c-1b7b-498a-b05f-98dcd95f2efd None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.621903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ee0c-1b7b-498a-b05f-98dcd95f2efd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.621903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ee0c-1b7b-498a-b05f-98dcd95f2efd None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.623301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 263/502] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:39.625098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ee0c-1b7b-498a-b05f-98dcd95f2efd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:39.629856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:39.630831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:39.631650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:39.631650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:39.631650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:39.631650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:39.631650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 240/503] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:39.632232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.730114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.735269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ee0c-1b7b-498a-b05f-98dcd95f2efd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.738273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f6ad-a2c7-4d93-a025-7f55718fec47 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.770696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.771293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.771564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.771858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.772566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:39.772865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.774248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.775235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.776067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.776743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.777025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.777828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:39.778138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.779793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f59220-73b9-42a2-b37d-5cebcf7c0a86 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.783504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 264/504] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:39.784114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.820257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 241/505] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:39.821555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.821847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.822122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.839520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:39.840765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:39.856043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:39.892890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679a4e4-d0b4-4827-856f-86af0443c4bd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:39.911845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 242/506] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:39.913744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.928158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:39.928776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:39.929045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:39.929316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:39.930159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:39.930471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:39.931965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:39.970372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 265/507] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:39.971211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:39.993238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.022403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052733c-eb7c-4070-ae83-1293a33f0349 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.060430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.061192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.061536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.061876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.062757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:40.063137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.064981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.076352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e741a49-576d-4c4d-9de7-76d124149ec0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.077855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 243/508] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:40.078682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.094310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.113672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.114225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.114506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.114773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.125247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.126889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:40.142713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:40.143945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:40.145575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:40.145575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:40.145575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:40.145575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:40.145575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 266/509] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:40.145575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.191553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851b7ad-e4e0-4974-adf9-cc46b6466a8b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.219346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.225662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.226322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.226622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.226890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.252473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633fc7b-8bad-42d4-a7ab-522556ae9680 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.289418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.290065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.290350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.290619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.295924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.297388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.301734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.303187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.309086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:40.319665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:40.320698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:40.321577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:40.321577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:40.321577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:40.321577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:40.321577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 267/510] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:40.321832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.405347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 244/511] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:40.406066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.429283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8330d73-96c2-442f-ba14-d31a45018b8e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.452990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.468297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.468915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.469167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.469457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.509540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cdc0e-5a45-4908-8680-769c2ab87f4b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.522362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.540158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.541643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.547280 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.548015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.548363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.548678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.557873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.557873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:40.569458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:40.570395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:40.571267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 245/512] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:40.571868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.648997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 268/513] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:40.649851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.676047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.688250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b20c38-931c-4f37-99bb-532fb29a9a87 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.722828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.723441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.723720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.724022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.724711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:40.725017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.726408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.742149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 246/514] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:40.743218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.757193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3db2e8-7ba5-4a5e-8d1d-ac7275c8949a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.778727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.797634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.798324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.798761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.799247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.800210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:40.800767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.802563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.814630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 269/515] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:40.815263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.853346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f09f0-edad-44fb-9e84-5df8b7156562 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.892558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.893703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.893703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.893882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.898408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:40.904255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.905723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.917955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7ead-cdcd-48ec-b490-51ba89264b55 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:40.921198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:40.922402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:40.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:40.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:40.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:40.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:40.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 247/516] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:40.923451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:40.957374 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:40.958071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:40.958422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:40.958788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:40.959727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:40.960221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:40.961862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:40.979859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 270/517] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:40.980682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.009501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.030783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1d0cd-51bd-4bdc-a066-03e63dcdd0cc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.073587 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.074235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.074510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.074778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.086491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.088004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.090959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08baaf-931f-475a-bf8d-813caffc149f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:41.102885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:41.103931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:41.104810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:41.104810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:41.104810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:41.104810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:41.104810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 248/518] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:41.104810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.128576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.129224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.129527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.129797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.136814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.138283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.140724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:41.152277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:41.153348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 271/519] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:41.154242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.214724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251416b-a8fe-48be-bad4-e3ead098a173 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.248025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.248641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.249226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.249501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.250487 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.252254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.260225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.268915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575ad1f-5c67-4438-99f7-10d9549c4a8a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.274689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 249/520] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:25:41 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:25:41.275465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.311895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.312529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.312803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.313074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.313808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:41.314157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.315654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.329479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 272/521] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:41.330667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.396525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.410917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115b62-a735-42e4-b5ef-c4dceb186d65 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.446660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a6828-eced-4265-8db5-9b81e2ea4bf3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.455855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.456516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.456799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.457114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.492730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.493270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.493541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.493801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.494636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:41.494933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.496661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.510153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 273/522] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:41.511009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.531522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.543866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.545501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.625402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817cd34-767c-4b25-b87f-bd2818cfec25 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.649894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.650681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 250/523] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:41.651427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.670879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.671533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.671846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.672120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.672974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:41.676323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.676323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.700524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 274/524] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:41.701651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.755584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.768926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39206b9-2da5-42d2-9779-d08e3d4d06a6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.808190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d7e6-bbf0-48e7-8524-d625ea6cbc7a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.813613 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.814260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.814531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.814797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.823732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.825641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:41.841218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:41.842117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:41.842944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 251/525] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:41.843544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.849859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.850519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.850793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.851064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:41.859276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:41.861141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:41.874247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:41.875141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 275/526] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:41.876875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:41.926644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:41.956489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c08d-8597-4518-8668-383cee4cac5c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.992316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07255772-8d82-4cf2-b16f-6544179c395b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:41.992875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:41.993344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:41.993739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:41.993994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.006045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.007614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:42.024080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:42.025257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:42.026096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:42.026096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:42.026096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:42.026096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:42.026096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 252/527] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:42.026548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.029085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.033291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.033925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.034202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.034469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.106006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.107472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.137251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab91ac-f5ad-453d-a4ee-b16d0f7723ff None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.150107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.174822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.175428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.175706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.176004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.176756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:42.177059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.178558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.196980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 253/528] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:42.197818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.228313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 276/529] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:42.229495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.267351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.308889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e534a1-8fb2-4123-ad6f-dca3806e6f0e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.346363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.346982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.347253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.347525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.348362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:42.348465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef8fb5-e2aa-4d48-b386-8c1576fdeb8b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.348661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.350226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.363826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 254/530] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:42.364739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.391808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.392425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.392741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.393009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.393753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:42.394530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.395930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.402947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.410670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 277/531] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:42.412847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.480100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de6a25-955c-4281-865f-300ccfb01735 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.504526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.519179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.519747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.520040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.520305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.521069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=dd1da21c00d8494587e3ede50a5a365b) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:42.521371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.522796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.528815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 255/532] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/role_assignments?user.id=dd1da21c00d8494587e3ede50a5a365b&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:42.529927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.530917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d2b15-a3f6-40a0-9fe2-7e2219165b6e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.582643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.583663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.584029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.584509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.601646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.604106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:42.622105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:42.623018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:42.624206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:42.624206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:42.624206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:42.624206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:42.626746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 278/533] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:42.627977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.683924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a70e48-1e3e-415c-88e8-36a585582a2f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.685938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.739786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.739786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.739786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.739786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.740293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=288250b6b648409aa179b3213321b869) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:42.740293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.740293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.740918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 256/534] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/role_assignments?user.id=288250b6b648409aa179b3213321b869&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:42.741797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.784162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4a457-89de-417b-ad8a-1baeadb42aaa None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.801558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:42.853198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f8d5-5262-4119-ba91-6faebb675a13 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:42.868761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.869429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.869708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.869978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.884675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.886161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.897383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:42.898013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:42.898288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:42.898553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:42.899257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:42.899559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:42.901008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:42.913868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:42.914774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:42.915591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:42.919322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 279/535] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:42.922698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:42.928955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 257/536] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:42.936077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.065575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05bce2-3b11-4341-98e6-827c780f1afe None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.083793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ec95e-a818-4c40-b334-7380f0a90f85 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.084330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:43.103810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:43.111724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.112361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.112637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.112906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.127128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:43.131925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.132723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.133136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.133561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.134447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:43.134903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:43.136510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:43.137781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:43.156644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 280/537] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:43.162670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:43.165750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:43.166786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:43.167867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 258/538] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:43.173752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.274203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176655c-516d-4ef4-8837-bb78957f0384 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.311310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.312218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.312672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.313176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.314388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=698e47aac57c48adb8836cec7226b039) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:43.314855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:43.316570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cdaea-dbaf-40db-a5cd-9ac9448048af None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:43.323265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 281/539] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:43 2022] GET /identity/v3/role_assignments?user.id=698e47aac57c48adb8836cec7226b039&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:43.370116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca47df9-2251-4755-a6a8-086670ab482c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.370661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca47df9-2251-4755-a6a8-086670ab482c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.371085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca47df9-2251-4755-a6a8-086670ab482c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.372991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 282/540] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:43.379225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.379735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.380199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.394722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825cf9-5262-4f10-9a0f-e9bc9ce64dd1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.402090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:43.403618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:43.435084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:43.482090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 283/541] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:43.492631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.493454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.493826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.494188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.495180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:43.495578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:43.497224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:43.508479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.515898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 259/542] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:43.517672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.552705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:43.627811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53918a5-dc39-4c3e-be5e-a7acb2037ee4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.636070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52c2e9-c0c5-4972-8c0a-b5b403e5cdf8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.676857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.677775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.678209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.678737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.678966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.679953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.680326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.680691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.681740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=d85bf2b530ce423d8f1c010b5044dba1) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:43.682153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:43.683867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:43.693103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:43.695954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 260/543] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:43 2022] GET /identity/v3/role_assignments?user.id=d85bf2b530ce423d8f1c010b5044dba1&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:43.698899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:43.699878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:43.700692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:43.703575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 284/544] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:43.712041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.838654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:43.847579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0124e-2255-4930-81df-83aa14c07307 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:43.896897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:43.897456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:43.897733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:43.897999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:43.898836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:43.899138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:43.900571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:43.910254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 261/545] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:43.916132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:43.949628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:44.038704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b4cf1-bdf6-47c8-bbbb-9e4ea71e31d6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:44.081654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:44.082301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:44.082573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:44.082895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:44.097786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:44.099245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:44.118327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:44.119234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:44.120107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:44.120107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:44.120107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:44.120107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:44.120107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 285/546] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:44.126490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:44.269161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91612650-909f-4be2-90de-c5393ee0f39b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:44.316540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:44.317179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:44.317454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:44.317722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:44.318421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:44.318719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:44.320121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:44.335006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 262/547] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:44.340540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:44.420895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:44.445553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e79144-d801-4da7-924a-98995ba2f8c2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:44.480002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:44.480643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:44.480915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:44.481184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:44.489297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:44.490752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:44.503004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:44.504012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:25:44.505401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:44.507920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 286/548] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:44.510394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:44.534815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:44.639830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154295ab-9901-4911-a0aa-f33d454958b1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:44.691920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:44.692546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:44.692819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:44.693085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:44.693896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:44.694193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:44.695641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:44.718291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 263/549] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 16:25:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:44.719971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:44.853305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0df1-1cb0-41d2-b06a-04d08e17144b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:44.895968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948451e-0870-4b8b-ac3d-05a4a9f297f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:44.896582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948451e-0870-4b8b-ac3d-05a4a9f297f3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:44.896850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948451e-0870-4b8b-ac3d-05a4a9f297f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:44.897115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948451e-0870-4b8b-ac3d-05a4a9f297f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:44.907330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:44.995474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948451e-0870-4b8b-ac3d-05a4a9f297f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:44.997061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948451e-0870-4b8b-ac3d-05a4a9f297f3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:45.127953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 287/550] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:45.136938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.179091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bca95-b8b4-4b9a-94ef-55896519ad67 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.179420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bca95-b8b4-4b9a-94ef-55896519ad67 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.179698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bca95-b8b4-4b9a-94ef-55896519ad67 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.181403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 288/551] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:45.187411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.187738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.187989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.213114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:45.214336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:45.248682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:45.284184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:45.297337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 289/552] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:45.324799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.340353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54da15-e886-4a69-a4d0-0491595764f5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:45.449276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:45.449860 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.450138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.450409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.469671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6be9c-c11e-4858-8441-0e10c7fa14cc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:45.480893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:45.482406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:45.508514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:45.509631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:45.510511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:45.510511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:45.510511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:45.510511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:45.510744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c116d-667f-426c-a7c3-856acdd825c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:45.510744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c116d-667f-426c-a7c3-856acdd825c7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.510744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c116d-667f-426c-a7c3-856acdd825c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.510949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c116d-667f-426c-a7c3-856acdd825c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.513172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 264/553] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:45.521477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.522741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c116d-667f-426c-a7c3-856acdd825c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ad9c116d-667f-426c-a7c3-856acdd825c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:45.527795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:45.528723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:45.529618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 290/554] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:45.532250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62563-4b1e-439e-b2cf-6e4be2df77b4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.532714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62563-4b1e-439e-b2cf-6e4be2df77b4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.533060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62563-4b1e-439e-b2cf-6e4be2df77b4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.534726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 291/555] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:45.540813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.541163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.541499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.561839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:45.563863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:45.587678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:45.657851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c311e9-0365-4a50-a7c9-8bdb83452bed None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:45.663849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 292/556] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:45.664971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.712748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:45.727120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:45.728848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.728960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.729198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.730869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:45.731427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:45.734051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:45.747720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 265/557] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:45.748700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.811778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64406afb-7a7c-426e-8e6e-4737d5762148 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:45.853805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:45.854867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.855394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.855931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.856898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:45.857445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:45.859129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:45.870336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 293/558] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:45.871521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.884284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc4cce-1537-4ed0-bd31-c89ec439e86f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:45.893168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:45.927077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:45.927740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:45.928050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:45.928310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:45.940105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:45.945464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:45.946873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 266/559] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:45.948077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:45.988509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74acb6e-59df-4e76-9753-933d8aaa5f99 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.003514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.028077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.028762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.029035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.029306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.037368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.038862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a99873a3-d15d-484e-82cb-2b0e8bbd781c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:46.051519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:46.052515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:46.053478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 294/560] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:46.054272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7bcf0-eab6-48ab-81c5-45eb5eb550f8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.054272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7bcf0-eab6-48ab-81c5-45eb5eb550f8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.054519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7bcf0-eab6-48ab-81c5-45eb5eb550f8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.059876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 295/561] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:46.059876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7bcf0-eab6-48ab-81c5-45eb5eb550f8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.075827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4ee90-b937-490e-92db-79a4345d77ac None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.113709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.114714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.115166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.115746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.127564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.129219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.142061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:46.145535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:46.146697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:46.147563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:46.147563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:46.147563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:46.147563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:46.147563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 267/562] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:46.147952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.170455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7bcf0-eab6-48ab-81c5-45eb5eb550f8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.209776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.210412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.210694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.210964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.211716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:46.212062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.213515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17685-faba-4847-8ff4-3709e56ac960 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.227983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 296/563] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:46.227983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.227983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.227983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.246503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:46.247616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:46.268673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.271863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb5d9-b2d0-4df7-965b-4b640d3c2bfa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.311835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.312790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.313075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.313553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.314740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:46.315053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.316590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.326357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 297/564] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:46.327865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.331032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 268/565] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:46.331939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.410603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.443238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b785afc-358f-4956-83df-3fd06d9bc05e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.444510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2ab04-e1d5-434e-ad5b-1e8ee653ae45 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.479468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.480134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.480424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.480690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.481396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:46.481713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.482014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.482654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.482925 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.483070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5c007-749d-4e76-96c1-2b65d2d51076 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.483164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.493539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.495023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.495441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 298/566] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:46.496723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59334f0d-0338-4753-8758-2b4d5770ad37 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.496952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59334f0d-0338-4753-8758-2b4d5770ad37 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.497239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59334f0d-0338-4753-8758-2b4d5770ad37 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.498462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 299/567] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:46.499477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59334f0d-0338-4753-8758-2b4d5770ad37 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:46.515802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:46.517029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:46.517906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:46.517906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:46.517906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:46.517906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:46.517906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 269/568] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:46.518186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.532195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.607309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59334f0d-0338-4753-8758-2b4d5770ad37 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.630303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36539a0-136e-4d8f-bbdf-7acdefd135c1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.637656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.646132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.646132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.646132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.646132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.646928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:46.647232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.648710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.654282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 300/569] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/role_assignments?user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:46.655057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.675304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeda76c-051e-47e5-88ad-f9eb18786fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.676371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeda76c-051e-47e5-88ad-f9eb18786fbb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.676835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeda76c-051e-47e5-88ad-f9eb18786fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.677269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeda76c-051e-47e5-88ad-f9eb18786fbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.695570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeda76c-051e-47e5-88ad-f9eb18786fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7aeda76c-051e-47e5-88ad-f9eb18786fbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:46.702892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:46.704126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:46.705179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 270/570] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:46.711079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.711079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.711079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.731633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:46.733071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:46.800476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bda61-b9f9-4b6f-a9a7-aada84f4878f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.817629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.822896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 271/571] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:46.824073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.845105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:46.845782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:46.846059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:46.846356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:46.855159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:46.856660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:46.869176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:46.870412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 301/572] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:46.871570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:46.941893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:46.961729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133794-0f2b-48ca-8865-9725c34a0bfe None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:46.993319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1b55e-2e3f-44c0-b982-9944cefe29a7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.015974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.016941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.017224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.017476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.018201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:47.018486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.019916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.036928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.037561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.037833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.038203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.038610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 272/573] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:47.039484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.046723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:47.047681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.050659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.069790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 302/574] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:47.069790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.114198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.167598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6440b3e-4339-4632-a77f-7587d4ff0a44 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.184635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09e980-30f4-46ab-9de7-09c5d0049e3b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.201118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.205398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.206054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.206331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.206600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.217351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:47.221882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.222504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.222783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:47.222965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:47.223875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:47.224701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:47.225317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.225317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:47.225317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.225977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.229318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 273/575] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:47.230095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.246078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 303/576] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:47.247323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.357869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bceb5d-1235-4926-8476-3cf9a2cf7cb7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.376112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e056a69-fc5b-4efb-a039-055dbed4c429 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.383017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.401997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.402741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.403079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.403392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.412028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.413427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.418411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.419046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.419413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.419693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:47.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:47.427550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 274/577] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:47.429640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.437723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.439473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:47.457069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:47.458244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:47.465602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:47.465602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:47.465602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:47.465602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:47.465602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 304/578] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:47.465602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.493609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.560438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76d05-c4c9-4adf-abe0-93cf79b6dfb9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.578070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b7cc-6173-4da6-853b-c2a7d0d04b98 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.598196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.598929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.599202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.599464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.600212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:47.600510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.602095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.616252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 275/579] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:47.616252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.617522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.618177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.618464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.618735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.631168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.632811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:47.649051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:47.650179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:47.651089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:47.651089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:47.651089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:47.651089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:47.655041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 305/580] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:47.655995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.688427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.760879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b288e-4356-4e06-bb24-ba0c1eeb8603 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.762015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.790199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04925-aa6a-4d4d-84a6-efa2b92c0863 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.801208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.801842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.802112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.802378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.803174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:47.803475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.804954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.823222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 276/581] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:47.823929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.839482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.840641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.841100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.841899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.843095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:47.843715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.845863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:47.861026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 306/582] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:47.866346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:47.934528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:47.938168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65084db2-a45a-424a-b9ac-15230ad45c31 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.977394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6fb8-6cb6-44b5-a1f3-aac667608f98 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:47.978572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:47.979426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:47.979923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:47.980283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:47.990941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:47.992441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:48.011303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.012781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.014651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:48.014651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:48.014651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:48.014651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:48.014651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 277/583] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.014651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.021877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.023288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.023745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.024211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.024763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:48.025075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.026466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.034917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.043952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 307/584] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:48.044857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.139806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a858eb2-1138-402e-9624-ece32ee8d15e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.153431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beaa161-a67a-4174-ab4a-28431054a5f2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.178616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e12fb-512c-497c-b925-54500da92466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.179578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e12fb-512c-497c-b925-54500da92466 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.180253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e12fb-512c-497c-b925-54500da92466 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.180623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e12fb-512c-497c-b925-54500da92466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.195260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e12fb-512c-497c-b925-54500da92466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.196911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e12fb-512c-497c-b925-54500da92466 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.199452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.200290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.200371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.200648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.209864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.211417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-258e12fb-512c-497c-b925-54500da92466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:48.213801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.214663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.215531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:48.215531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:48.215531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:48.215531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:48.215531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 278/585] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.215847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e12fb-512c-497c-b925-54500da92466 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:48.229822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.231105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 308/586] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.233126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.235364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.301241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.353899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb8b72-0d44-4c20-afd8-906bb7bb49ba None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.361857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e12fb-512c-497c-b925-54500da92466 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.393208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.394106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.394530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.394803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.399595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.400287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.400562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.400832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.401523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:48.401819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.403030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.403225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.405421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.422064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 279/587] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:48.422262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.423123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:48.424858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:48.425304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.425375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 309/588] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.425375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.493894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.540294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b83349-0e94-4106-90f0-8928ffc3a9e0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.549236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740412b-4f90-43f2-9d3c-fa5f42281c03 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.559346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.579235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.580162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.580432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.580687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.581483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:48.581779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.583432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.592544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.593176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.593463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.593738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.594573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:48.594893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.596297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.604044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 310/589] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:48.605365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.612105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 280/590] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:48.613272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.737403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2605d27-e549-4db2-b9b3-8fe902a825d6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.749881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8913010-9b40-4c21-8bc4-36316cc427b0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.764928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.779345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.780000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.780278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.780549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.781358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:48.781667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.783638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.787870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.788528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.788801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.789071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.796583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.797197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 281/591] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:48.798359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.798484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:48.812299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.813593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:25:48.814790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:48.815905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 311/592] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.816947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.856084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:48.920260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21a34e-c52b-46d6-8a61-19a99995ac24 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.929323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd4988-7ce4-4a96-b4b7-427ff8143311 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:48.959222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.960199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.960598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.961042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.968438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:48.969114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.969565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.969798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.973417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.974922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.976628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:48.978051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0f9d5d17-a11d-4d1c-9c6d-37208541d06e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:48.989170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.990182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:48.991063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:48.991493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:48.992317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:48.993212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:48.993212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:48.993212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:48.993212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:48.993460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 312/593] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.993460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893548-6c3e-40f6-888a-f48fb0a2a03e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:48.993460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893548-6c3e-40f6-888a-f48fb0a2a03e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:48.993460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893548-6c3e-40f6-888a-f48fb0a2a03e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:48.993697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 313/594] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:48.994358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893548-6c3e-40f6-888a-f48fb0a2a03e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:48.994500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 282/595] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:48.995468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.033400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.093491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.099903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e08fa-00b2-4cf5-b76d-a9f354b3f7f1 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.100421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893548-6c3e-40f6-888a-f48fb0a2a03e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.134557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.135858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.135858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.135858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.136488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:49.136787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.138247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.140028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8aebf-1b17-4075-b464-213a0ac7f08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.140739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8aebf-1b17-4075-b464-213a0ac7f08d None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.141017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8aebf-1b17-4075-b464-213a0ac7f08d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.141324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8aebf-1b17-4075-b464-213a0ac7f08d None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/dd1da21c00d8494587e3ede50a5a365b/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.156332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 283/596] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Aug 1 16:25:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:49.157418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.161891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8aebf-1b17-4075-b464-213a0ac7f08d None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=dd1da21c00d8494587e3ede50a5a365b, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.163368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8aebf-1b17-4075-b464-213a0ac7f08d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.178650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 314/597] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:48 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/dd1da21c00d8494587e3ede50a5a365b/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:49.180140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.180364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.180843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.202261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:49.203501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:49.299283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.308476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2be64-c1db-4168-8af2-a351dc842edd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.320134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 315/598] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:49.321141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.361520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.361921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.362329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.362636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.363368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:49.363751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.365302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.379098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 284/599] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:49.380354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.414242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.444961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8d8f2-6dda-489d-834d-badf8124febe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.486697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.487361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.487602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.487902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.490598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:49.490598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.490598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.507855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 316/600] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:49.507855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.513765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9efb14-f35c-48a6-a351-d3b5fa1747b2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.554240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.555234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.555511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.555825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.570234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.572220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:49.588681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:49.589812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:49.590825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:49.590825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:49.590825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:49.590825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:49.590825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 285/601] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:49.590825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.602566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.632201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82eee8-26f7-4de9-bc61-835737605bc9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.682384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.682384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.682384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.682384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.685985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.696813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:49.702095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:49.703084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:49.704375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 317/602] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:49.705129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.719654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2deb88-5cc0-4405-b7c9-50ca60f30376 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.758904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.760078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.760517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.760991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.768605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.770127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:49.782137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:49.783114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 286/603] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:49.784623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.837230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4ba45-ec42-4fea-ae67-e0793b580b95 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.865578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:49.885266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.886105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.886362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.886623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/288250b6b648409aa179b3213321b869/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.909558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e261bb-0910-494e-91fb-8862db935e57 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:49.911249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=288250b6b648409aa179b3213321b869, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.914026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.937285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 318/604] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:49 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/288250b6b648409aa179b3213321b869/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:49.939909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:49.963367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:49.964562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:49.965030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:49.965721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:49.966875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:49.967389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:49.969403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:49.985625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 287/605] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:49.986959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.008973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:50.082329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d243-c6b4-44f2-bb7f-6c1c6b6acc11 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.128532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.131102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.131781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.132251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.133176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:50.134303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.135934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.142026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c93c5-4baf-4cb4-a550-fbe35cce7ebd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.145976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 319/606] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:50.147421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.187091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:50.200999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.201637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.201915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.202187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.202948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:50.203253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.204853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.217816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 288/607] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:50.218592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.287729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab81b0-c34b-4e94-b730-d78875634641 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.298796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:50.325586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c806-7509-4dc8-b55e-1d45344a71d6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.331441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.332389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.332839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.333268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.342978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.345958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:50.361036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:50.362226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:50.363344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:50.366625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 320/608] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:50.367887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.381913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.384132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.384132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.384132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.397140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.399155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:50.415711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:50.416923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:50.418027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:50.418027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:50.418027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:50.418027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:50.423897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 289/609] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:50.424069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.466239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:50.492949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f195de-2efe-4f1a-8b2e-354af9d1caa3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.532223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.533456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.533841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109dab-fd0b-4c64-8456-159d5b0d4bee None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.534064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.534490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/698e47aac57c48adb8836cec7226b039/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.551436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=698e47aac57c48adb8836cec7226b039, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.553547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.571557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.572237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.572510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.572778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.573533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:50.573835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.575354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.576254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 321/610] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:50 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/698e47aac57c48adb8836cec7226b039/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:50.577503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.591729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 290/611] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:50.600199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:50.617268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:50.761196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:50.766002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b2a7-0bec-4f3b-82ab-32404d74bd1d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.789392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfa4bd-1cfb-450a-9ddf-0653bf14974f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:50.807150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.808384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.808384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.808916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/d85bf2b530ce423d8f1c010b5044dba1/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.829189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=d85bf2b530ce423d8f1c010b5044dba1, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.834110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.839577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:50.841202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:50.841547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:50.841821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:50.842719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:50.843025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:50.844659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:50.854154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 291/612] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:50 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/d85bf2b530ce423d8f1c010b5044dba1/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:50.862248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 322/613] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:50.868782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.056353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf122a-6c5b-4d29-b04b-e9ba0124f9bd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:51.103563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:51.104269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.104548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.104815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.113686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:51.115197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:51.128037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:51.129086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:51.130078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 292/614] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:51.135848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.237258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:51.269304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850356c-663f-4d5d-a5cb-5800f7e347ee None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:51.317546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:51.318176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.318397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.318624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.319466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:51.319795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:51.321343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76560fa-5f2b-4195-82dc-b5365892ebe1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:51.343526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 323/615] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 16:25:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:51.349176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.361079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:51.415652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dadaa3-6ada-40d2-b6fe-fdf490017798 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.416156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dadaa3-6ada-40d2-b6fe-fdf490017798 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.416625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dadaa3-6ada-40d2-b6fe-fdf490017798 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.423903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 324/616] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:51.427780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.428266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.428733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.455019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964516-2618-4052-806f-9d2755c3d787 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:51.458645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:51.460636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:51.489700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:51.490354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.490637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.490906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.507802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:51.507802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:51.520935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:51.522888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:51.524551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:51.524551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:51.524551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:51.524551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:51.524551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 293/617] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:51.528673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.588559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:51.618411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 325/618] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:51.633355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb98bea-6f68-474f-9a80-95f5b783a42f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:51.664729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.668476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:51.669107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.669386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.669658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.670446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:51.670730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:51.672221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:51.686431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 294/619] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:51.696235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.799703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dce47-bbe1-4611-8004-3dc7338e668a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:51.823857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:51.828360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:51.836760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:51.837414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.837692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.837973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:51.845167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:51.851706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:51.859736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:51.862178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:51.863945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:51.866933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 295/620] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:51.874467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:51.918724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0d2d0-2657-4394-8ac6-261aa27d5a64 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:51.996454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:51.997105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:51.997379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:51.997638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.021280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:52.032317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:52.034553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:52.036422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 326/621] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:52.046677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:52.095507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0098-0a58-493f-8033-c9d9d038700f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:52.167633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:52.168365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.168642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.168940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.169777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:52.170107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:52.171637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:52.194610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 296/622] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:52.200752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:52.238939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1e0ff-30fa-48ab-bc60-f844dba3c750 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:52.286465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:52.298553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:52.316492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:52.317131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.317410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.317680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.318514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:52.318919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:52.320338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:52.344358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 327/623] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:25:52.351068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:52.354849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef226d-27ec-495a-9619-736955da4184 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:52.395880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:52.396566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.396847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.397125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/4177b0f6e8204e5285ee4fdaeb2aa4c9/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.413874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=4177b0f6e8204e5285ee4fdaeb2aa4c9, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:52.415480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:52.438742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 297/624] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:25:52 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/4177b0f6e8204e5285ee4fdaeb2aa4c9/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:25:52.550503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a719b-0ee6-478b-a118-5583998a92d8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:52.593484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:52.594157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.594487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.594746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.608588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:52.610218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-468dc77d-92ad-4cc0-a987-bc27671d8929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:52.627514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:52.628524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:52.629744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:52.629744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:52.629744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:52.629744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:52.629744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 328/625] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:52.636257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:52.738796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047dfe3-4bb4-48f7-a337-48afdcc4aaf5 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.739119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047dfe3-4bb4-48f7-a337-48afdcc4aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.739332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047dfe3-4bb4-48f7-a337-48afdcc4aaf5 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.741541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 329/626] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:25:52.757050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.757368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.757595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.784973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:52.786179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:52.794032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:52.809789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1adc6-afe7-4da0-814f-dd8c4212caa6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:52.849407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:52.854909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:52.855541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:52.855872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:52.856147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:52.856847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:52.857153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:52.859644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:52.873111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 298/627] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:52.879690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:52.904217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 330/628] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:52.938776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:53.076012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f507c-ded7-413f-b10a-b70b27593972 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.117035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.117611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.117885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.118152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.133565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:53.135053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:53.150408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2063e5a-2315-4443-8afe-145a1983fbc8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:53.160827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:53.161856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:53.163273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 299/629] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:25:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:53.173003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:53.203797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:53.213494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da018-c8e3-4655-af83-224556952aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.215100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da018-c8e3-4655-af83-224556952aca None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.215941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da018-c8e3-4655-af83-224556952aca None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.216721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da018-c8e3-4655-af83-224556952aca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.249742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da018-c8e3-4655-af83-224556952aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-542da018-c8e3-4655-af83-224556952aca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:53.256501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:53.258320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:53.260040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 331/630] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:53.268250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da018-c8e3-4655-af83-224556952aca None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:53.330859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:53.352781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99367c-4f30-4e84-bc83-1e62be7e6d18 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.424168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da018-c8e3-4655-af83-224556952aca None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.441123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.441789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.442036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.442305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.443113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:53.443417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:53.445076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:53.473115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.474136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.474613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.475091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.476054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:53.476577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:53.478259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:53.488971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 300/631] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:25:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:53.490285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:53.497325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 332/632] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:53.506416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:53.581457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:53.619651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03519fbc-1a4b-4e71-a50c-dd5f38513546 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.624394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:53.637884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923667-12b4-40cd-8a63-ad0608584c4e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.661215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b07c3-fdc8-46c0-9458-46e06c045168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.661846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b07c3-fdc8-46c0-9458-46e06c045168 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.662162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b07c3-fdc8-46c0-9458-46e06c045168 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.662444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b07c3-fdc8-46c0-9458-46e06c045168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.688832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.688832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.688832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.688832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.699016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:53.700488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-dff8d227-f1e3-4025-b0a6-0474e74388bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:53.715678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:53.716701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:53.717593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:53.717593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:53.717593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:53.717593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:53.721863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 333/633] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:53.723104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154a7e1-863e-4626-a993-9947bf72695a None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.723385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154a7e1-863e-4626-a993-9947bf72695a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.723664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154a7e1-863e-4626-a993-9947bf72695a None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.725208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 334/634] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:53.726124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84be1b-7e4c-4b93-9b2b-0f59b9270bfe None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.726456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84be1b-7e4c-4b93-9b2b-0f59b9270bfe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.726727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84be1b-7e4c-4b93-9b2b-0f59b9270bfe None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.727969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 335/635] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:53.732044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84be1b-7e4c-4b93-9b2b-0f59b9270bfe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:53.761178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b07c3-fdc8-46c0-9458-46e06c045168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:53.762666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b07c3-fdc8-46c0-9458-46e06c045168 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:53.839251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:53.847322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84be1b-7e4c-4b93-9b2b-0f59b9270bfe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:53.883148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:53.883511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 301/636] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:53.884413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.884839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445767b-1a25-43b1-89db-a81166155241 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.885128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445767b-1a25-43b1-89db-a81166155241 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.885387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.885500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445767b-1a25-43b1-89db-a81166155241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.885903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.886605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:53.886910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:53.888367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298c4d7-ce18-4f94-8367-4bc49097b35e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:53.902908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 336/637] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:53.904003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:53.904287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:53.904559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:53.911030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c445767b-1a25-43b1-89db-a81166155241 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:53.912451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c445767b-1a25-43b1-89db-a81166155241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:53.914297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:53.936906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:53.937603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:54.001718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 302/638] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:54.002570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445767b-1a25-43b1-89db-a81166155241 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.033587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 337/639] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:54.034564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.123346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.129131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445767b-1a25-43b1-89db-a81166155241 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.149092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1656130-330d-42c1-8705-9d3a691c0218 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.170000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.170671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.170946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.171213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.182655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.184139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.192463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.193162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.193361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.193631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.195615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:54.199304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:54.200246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:54.201223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:54.201223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:54.201223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:54.201223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:54.202951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.204464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.205599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 303/640] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:54.206469 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:54.216818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:54.217772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:54.218712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 338/641] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:54.223697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.320614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8316c-d214-4472-84ba-14a09505332b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.332489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986b027-7edf-43e6-b4bc-d351b44de4ef None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.357349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.358005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.358278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.358547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.369577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:54.370589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.371228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.371504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.371796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:54.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:54.376706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:54.377607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 304/642] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:54.378349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.389251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:54.390961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:54.394970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:54.396057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:54.396936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 339/643] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:54.397646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.458031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.492618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba9dff-c6b3-4946-9fb8-44fd110c5d4f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.506271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480ac2-f510-4d54-b7d5-d6bc6f313e2d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.529239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.529886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.530134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.530426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.531085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:54.531382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.533081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.542123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.542752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.543026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.543295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.544131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:54.544432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.550510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 305/644] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:54.550674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.550772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.569997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 340/645] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:54.571190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.629919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.664945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fc0fd-7b44-4f6f-a513-8c257ce19b92 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.691110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a6507-922c-4ae2-bb9e-94343e12cfe0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.708939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.709426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.710063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.710337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.710606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.711324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:54.711771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.713359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.723827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 306/646] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:54.729743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.733522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.734155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.734432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.734779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.736462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:54.736672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.738202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.754718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 341/647] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:54.754718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.862557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378cba-4259-4c9f-9484-1ff416a4f2c7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.890353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ce62-15dc-46d0-b1b6-e08def16b585 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:54.899615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:54.906399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.907524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.907870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.908142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.915884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:54.917451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:54.930308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cc848-6a69-4cbb-b0f7-a520b4584465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:54.930546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:54.931559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:54.932511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:54.932945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cc848-6a69-4cbb-b0f7-a520b4584465 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:54.932945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cc848-6a69-4cbb-b0f7-a520b4584465 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:54.932945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cc848-6a69-4cbb-b0f7-a520b4584465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:54.933124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 307/648] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:54.933124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:54.981483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.012996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cc848-6a69-4cbb-b0f7-a520b4584465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.014769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cc848-6a69-4cbb-b0f7-a520b4584465 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.048665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5228-dc30-48d6-a912-295807752530 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.091785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.092443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.092725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.092999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.106942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.108454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:55.124089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:55.125055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:55.126051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:55.126051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:55.126051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:55.126051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:55.126051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 308/649] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:55.126762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.133904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 342/650] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:55.135267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231bbcb-34fd-4a91-bff5-7b360a2aa21d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.135556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231bbcb-34fd-4a91-bff5-7b360a2aa21d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.135856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5231bbcb-34fd-4a91-bff5-7b360a2aa21d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.137121 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 343/651] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:55.137808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231bbcb-34fd-4a91-bff5-7b360a2aa21d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.177905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.227623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.239236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943a042-eb2b-4bde-9422-5641efec56f9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.255491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5231bbcb-34fd-4a91-bff5-7b360a2aa21d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.283263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.284023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.284303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.284599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.297056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.301065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.308872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.309601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.309952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.310118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.310887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:55.313798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.315344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:55.318168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:55.319203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:55.320119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:55.320119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:55.320119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:55.320119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:55.320119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 309/652] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:55.320119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.335048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 344/653] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:55.336088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.439059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.446353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd824a-8cc4-4a4d-8cbe-2e0c6134ac91 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.451345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075c569-09af-432d-9fd0-21cb33061649 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.482163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.482810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.483089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.483364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.484098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:55.484198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.484402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.484807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.485082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.485352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.485937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390bcd8-8977-4a68-8af7-4c054a5880cd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.487363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.498091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 310/654] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:55.499065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.499635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.499950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.500222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.500440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:55.515896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:55.517114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:55.518025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:55.518025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:55.518025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:55.518025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:55.518025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 345/655] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:55.518781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:55.518886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.519092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:55.586804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 311/656] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:55.587656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.625642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca978e-bf7e-4c3d-b509-3b4dcb51eac5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.659221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.659861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.660134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.660403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.661087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:55.661469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.662772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.679845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 346/657] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:55.680702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.694179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.710270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2da1-84a1-4d61-a3f4-e93be1875522 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.735179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:55.749732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.750454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.750870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.751386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.752254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=dd1da21c00d8494587e3ede50a5a365b) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:55.752588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.754180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.760466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 312/658] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/role_assignments?user.id=dd1da21c00d8494587e3ede50a5a365b&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:55.761282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.799779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cb241-c3e1-4fe8-9a86-11715fa88c39 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.841901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.842554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.842832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.843126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.853648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.855265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:55.868857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:55.870106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 347/659] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:55.871045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.900561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c075729-8d6c-4707-9ab2-16382c589c4a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:55.945624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:55.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:55.946967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:55.947321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:55.948419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:55.948821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:55.950754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:55.967672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 313/660] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:55.968634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:55.982977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.001694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870dd10-8c16-41ef-a8aa-16335ac4dee7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.009417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.040802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.041449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.041719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.041995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.052913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:56.058225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:56.059159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:56.060041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 348/661] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:56.060729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.098489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fd7eb-9dff-416f-9457-6913fb335c69 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.137365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.138011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.138293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.138548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.147026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.148581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:56.163646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:56.164826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 314/662] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:56.165774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.189305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fcf19-f7bb-443f-b210-27ca0ab886bc None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.230013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.230676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.230984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.231166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.232003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:56.232346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.233800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.245684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.266418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 349/663] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:56.266418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.291818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.292080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48821b88-6f9f-4327-92bf-020c587df479 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.335536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.336233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.336511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.336781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.344070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.345620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:56.357364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:56.358307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:56.359174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 315/664] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:56.360408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.389820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358b935-accd-4121-b981-5ac25ded0101 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.431577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.431577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.431577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.431577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.432365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:56.432365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.432365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.446028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 350/665] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:56.446028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.557198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.578735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583b82b-3793-4af0-baba-76bcd5675fa7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.601589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43624ff0-4b53-44aa-adfb-f6c4a499ec24 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.617524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.618145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.618408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.618683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.638082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.644335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.645079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.645354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.645617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.646408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:56.646708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.648181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.665867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 316/666] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:56.666685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.703104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.705507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.784528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a322c5f-3f56-4e11-94c9-d24d27611969 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.794579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.812330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 351/667] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:56.813212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.827637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.828314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.828783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.829054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.830064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:56.830372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.831914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.843898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 317/668] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:56.844960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.897705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:56.924027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00331afd-95e7-416b-9169-f95a52bf0f86 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.953932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2179-cc82-413e-a335-0d76aea9d908 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:56.961965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.962626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.962892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.963148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:56.973813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:56.975235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:56.990863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:56.991998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:56.992923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:56.992923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:56.992923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:56.992923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:56.992923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 352/669] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:56.993252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:56.996411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:56.997046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:56.997320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:56.997584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.043868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.072298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.073675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.111449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15fa15-faf8-4609-aff0-6d6ac3200cfc None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.126359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.152493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.153617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.153931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.154252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.167347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.169033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.182903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 318/670] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:57.182903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:57.189810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:57.190704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:57.191592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:57.191592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:57.191592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:57.191592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:57.191592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 353/671] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:57.191592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.294460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eae1d-a103-4a5d-b4a2-f5b0223b3df0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.299251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.312223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4def3-10d4-4028-a2ba-b4f381b1a1cb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.341206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.341809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.342089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.342415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.343269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=288250b6b648409aa179b3213321b869) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:57.343586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.345575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.351080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.352024 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.352277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 319/672] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/role_assignments?user.id=288250b6b648409aa179b3213321b869&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:57.352699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.353148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.353375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.354436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:57.354934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.356627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.375844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 354/673] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:57.375844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.386254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.494908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd54ba-37d2-4f0e-8a16-66250ea74567 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.512896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feaaa-30de-400c-8007-d275d260d65c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.539301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.540353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.540863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.541340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.542373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:57.542913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.544703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.559716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 355/674] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:57.560687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.566415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.567358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.567884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.568414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.577946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.583265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.586093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:57.606787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:57.608308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:57.609255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:57.609255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:57.609255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:57.609255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:57.609255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 320/675] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:57.610117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.692967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.700248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f183d39-e3c5-4376-b752-98e6c6e7949d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.736752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.737399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.737775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.738050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.741508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d072-5849-4952-9988-36bd949c6ece None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.746193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.747643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:57.760634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:57.761638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:25:57.762566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:57.764841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 356/676] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:57.765892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.792742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.793816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.794197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.794517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.803506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.805290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:57.817208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:57.818136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:57.818995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:57.826033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 321/677] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:57.827232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.849487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:57.887809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50de8ee-0523-4e87-9fef-25cb4a5547b8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.927238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:57.928181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:57.928592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:57.928998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:57.930038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:57.930449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:57.931930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:57.945491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 357/678] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:57.946309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:57.962935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d7470-d170-4e85-8832-a659c9970780 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:57.988998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.004887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.005673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.006050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.006420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.007448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:58.007871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.009418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd3c0e-3ddf-4c7a-a3f6-5dbf6c4b4881 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.029507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 322/679] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:58.030937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e2474-b22e-4d6c-8df7-5c9bc247d4f9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.031319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e2474-b22e-4d6c-8df7-5c9bc247d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.031816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e2474-b22e-4d6c-8df7-5c9bc247d4f9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.033279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 323/680] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:58.033918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e2474-b22e-4d6c-8df7-5c9bc247d4f9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.065638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ffaa6-2c38-44df-a092-1b8721409a9c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.095055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.102274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.102967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.103311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.103644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.104478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:58.104844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.106334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.121701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 358/681] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:58.122533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.154121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e2474-b22e-4d6c-8df7-5c9bc247d4f9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.189629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.190270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.190629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.190798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.197879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.199977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a44bf8d4-fd68-44ed-a5df-2095620b318d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:58.211444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:58.212420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:58.213402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 324/682] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:58.213940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.214016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.214977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.233385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:58.234992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:58.238313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699616a-68eb-4175-a19c-ed7f8cfeccb6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.274936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.280893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.281549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.281825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.282118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.310673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 325/683] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:25:58.311908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.360969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.373541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.375042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.423815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba70e5e-cbd1-4192-8df4-84c9f17a67fd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.460587 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.461195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.461466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.461730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.462513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=d85bf2b530ce423d8f1c010b5044dba1) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:58.462946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.464416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.479839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 326/684] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/role_assignments?user.id=d85bf2b530ce423d8f1c010b5044dba1&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:58.479839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.497248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 359/685] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:25:58.498692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.517726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.602315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101461-6d08-42e5-962b-de5c9ab3ce84 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.608130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.614408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115ca94-2dd4-4dbd-b37a-578216091827 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.649074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.650621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.650900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.651176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.652197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:58.652317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.653802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.657440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.658564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.659083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.659752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.669166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 327/686] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:58.669867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.679779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:25:58.689979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:58.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 360/687] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:58.692180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.809856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.818786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d833-c19e-4a2d-bfd1-568d57f38f9e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.819896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f5b0f-aaae-4f0d-aafd-f9b79b030ae3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:58.859138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.860314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:58.860829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.861061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:58.861289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.861397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:58.861654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.861808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:58.862715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=698e47aac57c48adb8836cec7226b039) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:58.863128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.864947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.871716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 361/688] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/role_assignments?user.id=698e47aac57c48adb8836cec7226b039&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:58.872699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:58.884920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:58.886664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:58.920962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:58.925249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:58.926586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:58.927712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:58.927712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:58.927712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:58.927712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:58.933185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 328/689] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:58.944114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.039793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6a77e-34c9-46eb-93ee-f57dc755302c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.083176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.084233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.084510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.084778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.085515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:59.085818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.087224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.103787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 362/690] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:59.105863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.178654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639a1e0-620f-4206-b129-5cc4b65bc24f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.201551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:59.221637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.222681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.223208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.223679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.224654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:59.225277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.226960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.228897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d556e8-9e7f-4f69-80e6-7f584c3665ae None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.242791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 329/691] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:59.249085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.264012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:59.270294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.270950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.271227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.271497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.287290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.289240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:25:59.311035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:59.312048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:59.313768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:25:59.313768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:25:59.313768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:25:59.313768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:59.315728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 363/692] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:59.321439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.414660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b19c6-1468-4cb0-a17c-33c9623a883c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.483909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.484500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.484775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.485041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.498064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.501297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.505325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef541-5ccc-4256-9e73-fc14cd09e934 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:25:59.526289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:59.527270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:25:59.528202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:25:59.532149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 330/693] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:25:59.533876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.560166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:59.581104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.581741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.582010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.582279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.582980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:59.583277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.584743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47722397-496e-4b70-9dbc-86913b81a90f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.600814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 364/694] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:59.600814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac22f4-61b2-4787-a458-5eb994741484 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.600814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac22f4-61b2-4787-a458-5eb994741484 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.600814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac22f4-61b2-4787-a458-5eb994741484 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.600814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 365/695] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:25:59.606158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.606469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.606743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.625380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:59.631523 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:25:59.632966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:25:59.672826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bafca-a55f-4fbc-add3-acf8c8e134a0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.718789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.720629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.721162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.721838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.723290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:59.723998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.726084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.743894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 366/696] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:25:59.743894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.752582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 331/697] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:59.758549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.882306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a124ae6-8010-40df-b7d2-b01ef58a4284 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.905665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b779e6-566d-4fd8-895e-b8287e14b152 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:25:59.912130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:59.925482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.926833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.927214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.927581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.941673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.944238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.947087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:25:59.956443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:25:59.957159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:25:59.957506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:25:59.957840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:25:59.958695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:25:59.959092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:25:59.960685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:25:59.965688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:25:59.966676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:25:59.967613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:25:59.968220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 332/698] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/role_assignments?user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:25:59.968856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:25:59.970839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 367/699] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:25:59.981121 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:00.129087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74968fd9-6d61-4152-a502-9b32496ed3a2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:00.188350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee30f44-1b3c-4b95-8218-0584fa331b80 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:00.200183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:00.201252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.201731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.202587 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.217614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:00.220746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:00.227660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:00.228326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.228603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.228870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.230852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:00.231164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:00.232969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:00.240906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:00.241858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:00.242761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:00.242761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:00.242761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:00.242761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:00.244917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 333/700] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:00.251639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:00.263935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 368/701] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:25:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:00.265204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:00.304117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:00.305407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:00.368939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7f052-143c-4fa1-89e4-ab333ad2d218 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:00.379075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e297-1214-49e1-8eae-d14bf8e70c8b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:00.409646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:00.410385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.410738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.411106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.411945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:00.412352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:00.414201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:00.418780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:00.419411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.419681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.420010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.429513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 334/702] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:00.435436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:00.521296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:00.523980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:00.565133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:00.574449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e64ef3-06cd-40ce-9151-3089df102bdb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:00.578613 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:00.615234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965662ac-c9a9-461e-bcbd-38ce72296246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:00.616420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965662ac-c9a9-461e-bcbd-38ce72296246 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.616876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965662ac-c9a9-461e-bcbd-38ce72296246 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.617290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965662ac-c9a9-461e-bcbd-38ce72296246 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.618319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965662ac-c9a9-461e-bcbd-38ce72296246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:00.620214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965662ac-c9a9-461e-bcbd-38ce72296246 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:00.646254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 369/703] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:26:00.649772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:00.652446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 335/704] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:26:00 2022] POST /identity/v3/users => generated 254 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:00.835039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f7b6e-b2d1-4019-917c-1722ab43141e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:00.883366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:00.884052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.884334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.884627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.885808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3514-802d-435c-956e-9f649dab9c9d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.886798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3514-802d-435c-956e-9f649dab9c9d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.887410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3514-802d-435c-956e-9f649dab9c9d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.888360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:00.890346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 336/705] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:00.898163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:00.898872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:00.899326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:00.899542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:00.902589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:00.919131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:00.920346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:00.921351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:00.921351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:00.921351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:00.921351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:00.923669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 370/706] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:00.944396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:00.945979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:00.958084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.064694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 337/707] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:26:01.096333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.129637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba8b9-0393-47ed-938a-224f83b4af1e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:01.188367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:01.189711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.190000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.190282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.191276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:01.191664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:01.193678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:01.210063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 371/708] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:01.218172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.225670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:01.244317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13dd0d-8e44-4258-b0fc-575e349a4e88 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:01.295464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:01.296462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.296738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.297142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] PATH_INFO: `/v3/services/network` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.297912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:01.299477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:01.302636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:01.315336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:01.317301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:01.318608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 338/709] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:01.324827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.374794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796285f-26e5-46da-9bd4-4b7af1c4c134 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:01.434441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:01.435243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.435609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.435912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.446277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:01.447778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:01.462663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:01.464841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:01.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 372/710] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:01.473597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.595319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163a585-3cb5-4e6f-8e62-5d1a6afbdbbf None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:01.609726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:01.619512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf232-8a2b-42bb-88db-8fa2e6de4b21 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:01.642703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:01.643896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.644593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.645045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.646001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:01.646315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:01.647800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179974f2-5ce2-451c-ad09-9bb96432c15a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:01.667141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 373/711] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:01.668878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09601-b138-486b-8bde-f4a2a61efca4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.669030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09601-b138-486b-8bde-f4a2a61efca4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.669285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea09601-b138-486b-8bde-f4a2a61efca4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.671348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 374/712] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:01.672307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09601-b138-486b-8bde-f4a2a61efca4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.686377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:01.687317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.688001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.688770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.689687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] RBAC: Adding query filter params (name=network) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:01.690262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:01.692241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489cf2-8024-4229-8074-87ae86500854 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:01.700811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 339/713] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:01.703297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.704102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.704831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.726116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:01.727740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:01.751917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:01.810753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea09601-b138-486b-8bde-f4a2a61efca4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:01.834085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 340/714] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:26:01.834996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.868093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:01.869362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:01.869854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:01.870351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:01.871376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=dd1da21c00d8494587e3ede50a5a365b) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:01.871997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:01.874122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:01.881849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 375/715] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/role_assignments?user.id=dd1da21c00d8494587e3ede50a5a365b&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:01.883125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:01.930173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:02.068460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2243d-9fc2-4fa5-a2f1-77989bb3b47a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:02.114090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3ce3e-ba56-4c77-b3b8-f072b7265065 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:02.119753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:02.120862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:02.121347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:02.122055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:02.123979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] RBAC: Adding query filter params (type=network) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:02.124809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:02.127877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:02.140452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 341/716] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:02.168944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:02.170059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:02.170567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:02.171046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:02.171900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:02.174103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:02.190317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:02.192571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:02.194112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:02.199376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 376/717] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Aug 1 16:26:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:02.207131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:02.228734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:02.345669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9073fb3-0822-407e-8cfc-07fd06ceefbc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:02.416056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:02.417658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:02.418462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:02.419259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:02.420928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:02.421742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:02.424856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daeb4a1-25a0-4d52-9e46-fe9af9b30405 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:02.434378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 342/718] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 16:26:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:02.441699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:02.598576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:02.604504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11155-6b96-44c7-a163-ffdf7333708a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:02.664050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:02.664981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:02.665411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:02.665907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:02.666969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:02.667447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:02.670558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce531b87-98b2-4712-95bf-fa64e4bd4141 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:02.683808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 377/719] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 16:26:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:03.177568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2273c-38ab-42db-833a-1fc224bf3781 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.178123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2273c-38ab-42db-833a-1fc224bf3781 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.178123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2273c-38ab-42db-833a-1fc224bf3781 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.179702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 343/720] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:03.185229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c323b-15e4-4b95-8081-1dcc12f1a329 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.185483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c323b-15e4-4b95-8081-1dcc12f1a329 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.185749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c323b-15e4-4b95-8081-1dcc12f1a329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.204404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d44c323b-15e4-4b95-8081-1dcc12f1a329 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:03.205574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d44c323b-15e4-4b95-8081-1dcc12f1a329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:03.273655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 378/721] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:26:03.297351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2273c-38ab-42db-833a-1fc224bf3781 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:03.441543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:03.482760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2273c-38ab-42db-833a-1fc224bf3781 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:03.483602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7cba6-cc9e-43f3-8304-6d3021bbc840 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.483931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7cba6-cc9e-43f3-8304-6d3021bbc840 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.484198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7cba6-cc9e-43f3-8304-6d3021bbc840 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.487432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 379/722] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:03.491832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.492113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.492378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.528684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:03.531986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:03.536283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:03.536973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.537255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.537536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.562963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:03.574597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:03.575585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:03.576461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:03.580885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 344/723] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:03.582045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:03.635923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 380/724] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:26:03.676481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:03.716259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:03.794399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fef48c-dadd-49ad-9c04-e2fdc7281d8a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:03.831674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:03.832369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.832655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.832932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.844083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:03.849694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:03.850660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:03.851517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 381/725] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:26:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:03.856708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:03.861655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447a5e1-f8b9-4c16-a58d-81277f1304e0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:03.900249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:03.900875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:03.901156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:03.901427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:03.902157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:03.902458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:03.904795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:03.920058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:03.931878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 345/726] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:03.935028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.012262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:04.025701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0c586-0052-45b8-a9aa-a89ab76dfd71 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.061514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ae567-7474-44b7-8270-265f0906b3f9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.065043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.065848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.066130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.066505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.067390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:04.067707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.069138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.080006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 382/727] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:26:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:04.086595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.138389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.139471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.140011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.140536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.170138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.172151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:04.205758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:04.207438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:04.208540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:04.208540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:04.208540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:04.208540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:04.208540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 346/728] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:04.213135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.242673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b630000-90f7-47a4-8caa-312c8b21ba48 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.283177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.283862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.284140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.284414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.307082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.308620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.316114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:04.322910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:04.337938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:04.338949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:04.339928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:04.339928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:04.339928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:04.339928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:04.339928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 383/729] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:04.355519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.443032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3aed2c-05cc-4054-a01c-87c8fc281048 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.511373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.512034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.512312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.512581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.513277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:04.513577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.514959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.548444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 347/730] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:04.555790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.670409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b2325-f09f-4e00-9d98-7c1654183127 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.715534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.716545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.716889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.717237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.718142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:04.719073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.721157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecfc0c-a5b2-4c11-8cc8-e6fcdc80d8ed None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.726505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb986df-060e-4fc3-9e4d-74e48af43b3a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.736524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 384/731] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:04.738401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde51c7-443b-4369-b518-14bf6dd90308 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.738768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde51c7-443b-4369-b518-14bf6dd90308 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.739131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde51c7-443b-4369-b518-14bf6dd90308 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.741083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 385/732] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:04.741866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde51c7-443b-4369-b518-14bf6dd90308 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.748928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:04.780789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.781608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.781886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.782164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.792991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.792991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-03b18cbe-8a8e-46fd-9fb9-265170f1d617 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:04.807504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:04.808582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:04.809580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 348/733] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:04.810707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4ea85-69ae-47c4-8ed5-e271fd903952 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.810707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4ea85-69ae-47c4-8ed5-e271fd903952 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.810858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4ea85-69ae-47c4-8ed5-e271fd903952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.820101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:04.830386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-dac4ea85-69ae-47c4-8ed5-e271fd903952 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:04.831750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dac4ea85-69ae-47c4-8ed5-e271fd903952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:04.873834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde51c7-443b-4369-b518-14bf6dd90308 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:04.908434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 349/734] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:26:04.913859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b08c-91fa-4068-af3a-ce4a280a0fa2 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.914562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b08c-91fa-4068-af3a-ce4a280a0fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.914962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:04.915251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0b08c-91fa-4068-af3a-ce4a280a0fa2 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.915621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.915932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.916257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.919365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 350/735] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:04.920419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b08c-91fa-4068-af3a-ce4a280a0fa2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:04.930171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:04.931681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9f3dfeb6-d4f4-4b06-b24d-5b7d3d321183 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:04.944788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:04.946640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:04.948487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:04.950887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 386/736] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:26:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:04.951969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:04.953519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:04.953975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:04.974697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:04.975919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:05.050353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.055472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0b08c-91fa-4068-af3a-ce4a280a0fa2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.063450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 387/737] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:26:05.065007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.101839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.102627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.102906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.103177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.104010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:05.104316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.105776 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.122700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.128201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 351/738] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 16:26:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:05.129015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.199169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1d719-e785-4296-b59e-633931ee1abe None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.241797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212a42b-f9d7-4aaf-a356-44a004c5d28f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.243623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.244311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.244582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.244905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.259419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:05.264835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:05.266274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:05.267269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 388/739] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:05.269120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.292657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.293026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.293212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.293472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.294461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:05.294767 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.296407 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.319151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 352/740] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:05.319950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.344976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.397094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0157eb-b3fa-4197-8b5c-c5fab39bbdcb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.400805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.447610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.448514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57691e-47b5-4e10-bb87-e45d6f348887 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.448625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.449006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.449432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.450536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.452840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.475184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 389/741] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:05 2022] POST /identity/v3/services => generated 240 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:05.476275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.513135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715a7ed-def7-496c-ac57-c5096f89cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.513980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a7ed-def7-496c-ac57-c5096f89cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.514376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a7ed-def7-496c-ac57-c5096f89cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.514745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a7ed-def7-496c-ac57-c5096f89cfe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.609956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715a7ed-def7-496c-ac57-c5096f89cfe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.611698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715a7ed-def7-496c-ac57-c5096f89cfe1 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.613773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163c37-523c-4274-a260-e3ad5bc12545 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.629240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.659750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.660488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.660765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.661034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.661760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:05.662091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.663429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.677837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 390/742] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:05.677837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.712174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.743014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 353/743] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:26:05.744359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c319a-67ca-4d84-b0c3-8f22edd5125b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.744641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c319a-67ca-4d84-b0c3-8f22edd5125b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.744914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c319a-67ca-4d84-b0c3-8f22edd5125b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.750941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 354/744] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:05.750941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c319a-67ca-4d84-b0c3-8f22edd5125b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:05.810058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c608d-5454-495d-8b4d-7b9cff85f24e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.847010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afe512-1f0e-4641-86d0-a6ae2f0d7325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.847638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afe512-1f0e-4641-86d0-a6ae2f0d7325 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.847938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afe512-1f0e-4641-86d0-a6ae2f0d7325 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.848221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afe512-1f0e-4641-86d0-a6ae2f0d7325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.896535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c319a-67ca-4d84-b0c3-8f22edd5125b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:05.912769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.924590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afe512-1f0e-4641-86d0-a6ae2f0d7325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.926145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afe512-1f0e-4641-86d0-a6ae2f0d7325 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.937561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:05.938945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:05.939359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:05.939797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:05.953484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:05.955500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:05.956154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:05.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:05.973637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:05.974865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:05.974865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:05.974865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:05.974865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:05.976566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 355/745] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:05.977539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.045249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 391/746] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:26:06.047147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.047430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.047700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.066986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:06.068246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:06.109887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e053-e74a-4653-92a4-fbfaba432e04 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.146433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 392/747] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:06 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:26:06.147495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.158609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.159500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.159992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.160402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.174063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.176260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:06.194030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:06.195259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:06.196585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:06.196585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:06.196585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:06.196585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:06.200063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 356/748] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:06.200779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.222091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:06.223880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:06.280778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628df1fb-a62a-4887-b9db-ade708dbcf16 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.321800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e6aa7-8693-42b8-9122-37759f092243 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.325947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.326580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.326853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.327120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.327872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:06.328233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.329920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.343412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 393/749] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:06.345031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.367988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.368675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.368952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.369218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.383980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.385439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:06.401435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:06.403194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:06.404861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:06.404861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:06.404861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:06.404861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:06.404861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 357/750] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:06.404861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.472995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710c95-3956-45aa-8599-32c1fc8b5851 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.486112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:06.492943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:06.512776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.513732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.514025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.514277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.514973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.517304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.529768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8cd3c-a57d-4d3e-8fab-8a6ec36499d7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.538266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 394/751] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:06 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:06.539529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.588548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.589672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.590677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.591198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.592334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:06.593377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.596750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.617723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 358/752] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:06.619254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.689876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5143965-8433-4199-b2cc-ecfa81b9f701 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.736771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.737217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.737611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.737999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.748681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.751243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:06.763956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:06.764972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:06.766122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:06.767433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 395/753] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:06.767871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.770842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57e6d5-393a-4e95-99f5-563a4a48461c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.799189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:06.820045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:06.821963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.822868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.823015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.823254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.824045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:06.824345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.826231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.850255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 359/754] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:06.851713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:06.915702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dd500-d6be-42a2-8b6c-d41ae13f3adb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.959809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:06.960955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:06.961514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:06.961975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:06.970459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:06.972438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:06.985746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:06.986692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:06.987579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:06.989259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e2276-ad87-4ce3-ada1-6e4ddd0d09da None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:06.990389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 396/755] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:06.991495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.029590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.030476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.030857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.031288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.032899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:07.033414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.035065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.056781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 360/756] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:07.057869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.105160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:07.111793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:07.114520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8156c25-2c65-4ed2-9b92-9002f45f5b5b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.155223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.156132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.157186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.157565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.168159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.169748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:07.183045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:07.184041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:07.184945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:07.187083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ad544-6c8f-488f-a45b-36c7f5cb574c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.190681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 397/757] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:07.191521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.242609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.244029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.244642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.245200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.246139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:07.246848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.249634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.267678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 361/758] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:07.268968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.313219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd78942-ebe1-4d74-8e3b-c00edebe4d5e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.356268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.357378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.357965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.358326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.371004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.372738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:07.389977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:07.391323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:07.392451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:07.392451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:07.392451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:07.392451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:07.393008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 398/759] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:07.393969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.411870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d7c40-c83e-4f01-957e-e4654c49ed03 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.412905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:07.422514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:07.471511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.473137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.473527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.473900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.475071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:07.475483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.477864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.507142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 362/760] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:07.507142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.535324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eaed-1f57-4e18-b388-80be209ea4e4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.574490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.575112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.575388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.575726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.576562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:07.576918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.578972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.591875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 399/761] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/role_assignments?user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:07.592903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.624014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab23dd5-db89-4826-8e42-9b8268503b19 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.665968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.672987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.679463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.679981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.680959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:07.681495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.683225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13ca76-0979-48a8-8c29-d7e5d285cbcc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.693586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:07.698909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 363/762] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:07.700483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaacd7-b412-48d2-8588-38d34b8d2bb7 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.700915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaacd7-b412-48d2-8588-38d34b8d2bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.701296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaacd7-b412-48d2-8588-38d34b8d2bb7 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.707036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 364/763] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:07.708965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaacd7-b412-48d2-8588-38d34b8d2bb7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.739146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:07.742273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0cf9e-b362-4f2f-9839-3e481970d5ce None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.786587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.787396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.787745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.788132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.789297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=5a83ef8e97ae4500943dfbcdd3fb7587) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:07.789706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.792259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339484b3-bb74-487c-90d6-a043e534918a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.803870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 400/764] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/role_assignments?user.id=5a83ef8e97ae4500943dfbcdd3fb7587&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:07.805628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.806108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.806517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.832666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:07.834023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:07.846951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaacd7-b412-48d2-8588-38d34b8d2bb7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:07.893662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:07.894526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:07.895715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:07.896125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:07.920761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:07.922586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:07.934343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 401/765] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:07.949919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:07.951079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:07.952093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:07.958623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:07.959140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 365/766] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:07.966390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:08.040215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:08.082022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3397b96-45af-4eaa-853c-873388ef7409 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.122034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.122692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.123282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.123553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] PATH_INFO: `/v3/services/network` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.124238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.126376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:08.136923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:08.138463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:08.139341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 402/767] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:08.145189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:08.260304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde321b-0a69-4e8c-8bca-985ea1690c50 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.265214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:08.297346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:08.301397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.302183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.302466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.302743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.303435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] RBAC: Adding query filter params (name=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:08.303742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.305717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.312761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 403/768] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:08.319542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:08.321564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734c283-015c-47c2-9b1d-5b611d1844fb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.435724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4f769-6fbc-4725-990a-31810c7f1c50 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.442926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.443582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.443931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.444206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.444987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:08.445295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.446862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.474955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.475576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.475834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 366/769] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:08.475960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.476139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.476824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] RBAC: Adding query filter params (type=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:08.477139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.478593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.482711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:08.484401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 404/770] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:08.490870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:08.619336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:08.686007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce10e-5ad9-42c4-adee-3db1e239ad12 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.722026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837347e1-b90d-4ce7-874a-766f8fa595ed None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.738238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.741917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.741917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.741917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/dd1da21c00d8494587e3ede50a5a365b/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.745118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=dd1da21c00d8494587e3ede50a5a365b, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.746586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.772522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 367/771] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:26:08 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/dd1da21c00d8494587e3ede50a5a365b/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:26:08.785780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.786452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.786738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.787011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.787796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] RBAC: Adding query filter params (interface=public, service_id=4891f843d37b44d28c64422398313a31, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:08.788118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.789619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0aa5b-53ce-46f9-bd7f-577c72051060 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.800810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 405/772] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:08 2022] GET /identity/v3/endpoints?service_id=4891f843d37b44d28c64422398313a31&interface=public®ion_id=RegionOne => generated 194 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:08.806681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:08.809592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:08.943411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feb9e4-36c1-4bff-b0f0-145ee4f927df None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:08.983476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49633b-eca1-4a24-b024-2eb0e6b9414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:08.984178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49633b-eca1-4a24-b024-2eb0e6b9414e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:08.984471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49633b-eca1-4a24-b024-2eb0e6b9414e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:08.984919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49633b-eca1-4a24-b024-2eb0e6b9414e None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:08.985885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49633b-eca1-4a24-b024-2eb0e6b9414e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:08.987419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49633b-eca1-4a24-b024-2eb0e6b9414e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:08.993763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 368/773] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:08 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:09.007283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5c1a-1e5b-4e2f-8d4b-4c491dc7534f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:09.007598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5c1a-1e5b-4e2f-8d4b-4c491dc7534f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:09.007862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d5c1a-1e5b-4e2f-8d4b-4c491dc7534f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:09.009411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 406/774] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:09.015468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:09.015877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:09.016363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:09.041100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:09.042914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:09.146430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:09.160189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 369/775] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:09.186703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5c1a-1e5b-4e2f-8d4b-4c491dc7534f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:09.299278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d5c1a-1e5b-4e2f-8d4b-4c491dc7534f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:09.324288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:09.337011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:09.337664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:09.337989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:09.338268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:09.338899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:09.340306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:09.349900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:09.350872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:09.351704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:09.352526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 407/776] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Aug 1 16:26:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:09.358874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:09.489320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dfe91-fdde-4ab4-b2e4-70015cf07aa9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:09.551678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:09.552390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:09.552835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:09.553315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:09.554435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:09.554824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:09.560456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:09.566878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 370/777] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 16:26:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:09.572935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:09.723561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d651-b7f5-4213-b1f6-1ec003821fc6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:09.772359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:09.772996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:09.773273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:09.773543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:09.774228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:09.774531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:09.776007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:09.788878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 408/778] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 16:26:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:09.794892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:09.880773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:09.931608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859138a5-1cb5-4eca-9b90-bd70615f0968 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:09.977551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:09.978184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:09.978466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:09.978752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:09.979552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] RBAC: Adding query filter params (interface=public, service_id=d161e70f3cb4417ea27f403a3063a247, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:09.979919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:09.981312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad644b-bb87-4efa-96ed-c9f35fa58a7f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:09.990059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 371/779] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:09 2022] GET /identity/v3/endpoints?service_id=d161e70f3cb4417ea27f403a3063a247&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:09.995503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:10.069603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:10.137563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c7731-1fba-4810-b210-98d108efd1be None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:10.184742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c867496-f743-4884-9a4f-66693e556a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:10.185391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c867496-f743-4884-9a4f-66693e556a4b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.185674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c867496-f743-4884-9a4f-66693e556a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.185947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c867496-f743-4884-9a4f-66693e556a4b None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.186689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c867496-f743-4884-9a4f-66693e556a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:10.188278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c867496-f743-4884-9a4f-66693e556a4b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:10.201331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 409/780] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:09 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:10.218815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b939a88-0d6f-4b5a-ae50-f1be4cee61fe None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.219160 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b939a88-0d6f-4b5a-ae50-f1be4cee61fe None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.219429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b939a88-0d6f-4b5a-ae50-f1be4cee61fe None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.221332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 372/781] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:10.227182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d10f-bbee-4ed3-9d65-59ebcc62658e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.227474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d10f-bbee-4ed3-9d65-59ebcc62658e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.227791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d10f-bbee-4ed3-9d65-59ebcc62658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.250301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-79d7d10f-bbee-4ed3-9d65-59ebcc62658e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:10.251618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-79d7d10f-bbee-4ed3-9d65-59ebcc62658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:10.345145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 410/782] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:10.376572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b939a88-0d6f-4b5a-ae50-f1be4cee61fe None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:10.536265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b939a88-0d6f-4b5a-ae50-f1be4cee61fe None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:10.564284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:10.579127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:10.579896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.580178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.580526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.585423 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc342d-58a6-41db-9ad7-4c7121506066 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.586066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc342d-58a6-41db-9ad7-4c7121506066 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.586556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc342d-58a6-41db-9ad7-4c7121506066 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.588719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 411/783] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:10.594932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.595407 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:10.595795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.596287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:10.602919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:10.603965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:10.605283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 373/784] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:10.612156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:10.626530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:10.628314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:10.712134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:10.777786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 412/785] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:10.791233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4abcb8-4cd8-4259-8ea4-fdb7616922dd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:10.812361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:10.858757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:10.859456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:10.859783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:10.860058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:10.860778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:10.861079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:10.862461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:10.872308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 374/786] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:10.878506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:10.974959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.002935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a7fe2-1de4-4db6-be31-9aa837090e72 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.037206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c3f81-3bd9-41f8-8069-6369b158d0e6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.039405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.040301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.040683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.041056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.053000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:11.059166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:11.060408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:26:11.061469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:11.064447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 413/787] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:26:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:11.070626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.113071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.113710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.113829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.113996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.114260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.144701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.146192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:11.181147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:11.182131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:11.183248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:11.183248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:11.183248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:11.183248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:11.183248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 375/788] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:11.188857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.197535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2b5c8-9257-4fbe-ad59-d5fcf8528630 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.247197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.248545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.248927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.249265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.250329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:11.251052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.252599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.262962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 414/789] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:11.270349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.346128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.350366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1bf45-8266-4fe1-87dc-28f9751172d4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.386923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.387827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.388246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.388697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.389415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:11.389691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.391128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85b2d7-a3cd-48e3-95d3-a0d5840c61a8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.408486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 376/790] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:11.410003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947d03-2267-46a5-bf43-e5a178abcee3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.410287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947d03-2267-46a5-bf43-e5a178abcee3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.410569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947d03-2267-46a5-bf43-e5a178abcee3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.416212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 377/791] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:11.416212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947d03-2267-46a5-bf43-e5a178abcee3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.432568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.439545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6d0da-a2ff-46be-b92e-6caa905d6b3b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.486907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.487568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.489744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.490028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.504084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.504801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d0ef8016-fc87-4119-88d3-efff47fff7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:11.523985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:11.524992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:11.525874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:11.525874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:11.525874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:11.525874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:11.525874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 415/792] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:11.526807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.527093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.527359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.548292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947d03-2267-46a5-bf43-e5a178abcee3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.551241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:11.552261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:11.597543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.598555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.599185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.599755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.609544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.612963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:11.629806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:11.630822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:26:11.631812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:11.634579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 378/793] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:11.635592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.647960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 416/794] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:11.647960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.684351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.754515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.774919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b143bd8-033d-4c8b-bf4a-5188cb566bc2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.781860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0aed4a-0bac-4901-aa29-dcf63589b042 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.814500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.815128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.815405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.815676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.816418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:11.816708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.818094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.826135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:11.826950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.827254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.827553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.828663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:11.829008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:11.830733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d8f5-5bcd-4ea3-89f8-bb5ab9c42f8f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:11.836078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 379/795] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:11.836078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.860287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 417/796] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:11.860287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362a8c0-00ef-4a8f-b548-3e71da5560aa None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:11.860287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362a8c0-00ef-4a8f-b548-3e71da5560aa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:11.860287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362a8c0-00ef-4a8f-b548-3e71da5560aa None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:11.860287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 418/797] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:11.860287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362a8c0-00ef-4a8f-b548-3e71da5560aa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:11.933710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:11.958787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d864-2789-4cd1-a3fc-0d4cfafddb13 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:11.978421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362a8c0-00ef-4a8f-b548-3e71da5560aa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.000066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.000755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.001071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.001395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.012931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:12.018320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.019287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:12.020158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 380/798] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.021292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.029258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.030177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.030532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.030955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.035059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.041939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.044094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-a7039960-4c78-4818-9e46-62545ca6796b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:12.059507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.060528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:12.061467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 419/799] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:26:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.063611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.063611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.063611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.086348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:12.088104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:12.138694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ca205-b29f-4ba9-a7db-c4604bcb1d44 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.168762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 420/800] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:12.169564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.184808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.185883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.186372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.186847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.195645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.200992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.202560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:12.220994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.221950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.222836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:12.222836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:12.222836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:12.222836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:12.222836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 381/801] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.223551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.283500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee48df4-9055-458c-8d91-f84ff8207bf7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.309796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.319350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.320075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.320351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.320619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.321336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:12.321638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.323004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.331511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 421/802] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:12.332375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.344155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c3a29-ec53-4ca7-b21d-a32f09ccc1e6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.381826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.382448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.382726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.382996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.383813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:12.384122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.385750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.410173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 382/803] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:12.410173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.455930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ceafd-49ca-489b-a588-c69758763021 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.468612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.491790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.492436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.492740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.492983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] PATH_INFO: `/v3/services/network` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.493607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.495015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.505047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.506372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:12.508110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 422/804] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.509391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.545369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90389a-a146-4b1b-a3e2-4bcfaec3574f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.566265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.583388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.584145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.584565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.584901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.585700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:12.586043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.587448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.600717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 383/805] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:12.601557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.638172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c4f24-c948-4371-84ec-bdedc66dfb57 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.687534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.688424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.688716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.689099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.705011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.706578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:12.721747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.722947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.724224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:12.724224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:12.724224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:12.724224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:12.729129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 423/806] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.730067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.732019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ca00-2bad-48c0-ae79-3a25e30ca989 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.743282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.771544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.772225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.772497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.772763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.784823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.786298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:12.802419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.803459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.804389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:12.804389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:12.804389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:12.804389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:12.804389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 384/807] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.805132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.844388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159851f-2c0c-4d8a-a67d-42d392feea87 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.848667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:12.885052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.885666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.885917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.886182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.886940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] RBAC: Adding query filter params (name=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:12.887257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.888774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.894585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 424/808] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:12.895477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:12.918988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0326cc2-04c7-40ad-b2f2-f216a81e59de None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:12.955167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:12.955718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.956026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.956296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.963692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:12.965389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-89f01f3b-6dc5-4099-8306-6b508041df1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:12.977635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:12.979220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 385/809] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:12.980147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a44a4-1aad-4b1b-9c62-63b218aa6b4d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:12.980690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a44a4-1aad-4b1b-9c62-63b218aa6b4d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:12.980690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a44a4-1aad-4b1b-9c62-63b218aa6b4d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:12.986083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 386/810] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:12.986083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a44a4-1aad-4b1b-9c62-63b218aa6b4d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.007381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.013682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cb02-7d2c-494e-9eee-dcf2a9c74862 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.053067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.053997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.054367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.054749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.055550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:13.056164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.057826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.072955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 425/811] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:13.073632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.104140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.104515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a44a4-1aad-4b1b-9c62-63b218aa6b4d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.141856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.142484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.142770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.143159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.143803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:13.144106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.145490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.159412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 387/812] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:13.160481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.186625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293508c-9924-4c46-8fa2-2c2b70faae51 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.225067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.225601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.225877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.226153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.226831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] RBAC: Adding query filter params (type=network) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:13.227133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.228541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103148d-2e9f-4305-b3a1-fd51e6b0f404 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.233969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 426/813] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:13.235023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.235305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.235581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.255078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:13.256321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:13.256484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.277816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65e6b4-d801-4695-a90b-ffca7669d6e7 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.319995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.320628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.320901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.321174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.321932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:13.322233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.323678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.330649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 427/814] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:13.332153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.337021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 388/815] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:13.337842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.366097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.454275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd81c6-b562-4bb4-a6bb-3dd097ccd153 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.456358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11e113-86f1-41cd-9d3d-bb98eb806aa7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.489532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.490091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.490364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.490632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.494873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.495622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.495998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.496343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.498135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.499884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.502652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.506411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.508113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:13.512676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:13.514007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:13.515002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:13.519411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 389/816] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:13.520371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:13.525725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:13.526894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:13.527937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 428/817] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:13.528536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.624539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.635742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd98d-515d-4b0f-9ccc-c6d986d05d04 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.645199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cfc93-ac3f-4070-b1b8-f05a36520774 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.680622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.681607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.682066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.682581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.683748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.686199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.694543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.695296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.695574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.695926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/4177b0f6e8204e5285ee4fdaeb2aa4c9/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.698127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=4177b0f6e8204e5285ee4fdaeb2aa4c9, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.699633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.715095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 429/818] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:26:13 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/4177b0f6e8204e5285ee4fdaeb2aa4c9/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:26:13.715732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.735812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 390/819] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:13.735812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.810096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.861931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5a2ee-4b7d-45a4-8470-59e5c70189c2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.878406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebce84-4bff-4b21-93aa-42d31624a1ef None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:13.907877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.908791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.909768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.910201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.911414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.913549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:13.924540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:13.925524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:13.926353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 430/820] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:13.927185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:13.931253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:13.931978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:13.932224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:13.932638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:13.933564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:13.933880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:13.935525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:13.940189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:13.952240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 391/821] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:13.953671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.057178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde358db-e8fa-4f99-bd48-8dc0dcd508f8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.083632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e022e-b3ca-4e5f-8843-04a768383966 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.097104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.097770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.098066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.098334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.099133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:14.099486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.101120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.105080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:14.129574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 431/822] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:14.129574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.143490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.144232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.144739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.144948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.145636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:14.145938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.147406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.159588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 392/823] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 16:26:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:14.159588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.218402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:14.271360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735270b4-915f-42e2-a059-97df1dd0fa23 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.282545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd860f-bbc5-4705-acbb-0ed2b0093a14 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.314965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.316055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.316580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.317142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/5a83ef8e97ae4500943dfbcdd3fb7587/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.320773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.321401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.321678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.321950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.340853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=5a83ef8e97ae4500943dfbcdd3fb7587, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.342229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.356728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 432/824] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:26:14 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/5a83ef8e97ae4500943dfbcdd3fb7587/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:26:14.357601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.397012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:14.426620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.428171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.519465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb9243-1d89-4906-8433-dea155673349 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.536487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:14.551057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 393/825] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:14.557008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.566448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.567446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.567944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.568458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.569390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:14.569924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.571834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.578944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 433/826] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Aug 1 16:26:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:14.584454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.667144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:14.700819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ebcce-6c7f-439a-987a-6559bca85b1f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.719557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6031-8adb-45ae-8ac1-86e5276078d6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.747165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.748324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.748947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.749308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.763456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.763855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.764671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.765264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.765390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.765879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.767016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.768873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:14.784366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:14.785454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:14.786349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:14.786349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:14.786349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:14.786349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:14.786349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 394/827] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:14.791515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:14.797612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 434/828] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:14.912954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0883946-db61-4a38-9a46-50be49e9d495 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:14.946792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:14.961804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:14.962431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:14.962742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:14.963022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:14.963713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:14.964045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:14.965599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:14.980919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 395/829] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:14.988453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:15.027424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:15.097568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3845b-3ebc-4cec-bebe-4df12680b1a5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:15.134518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:15.135163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:15.135442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:15.135713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:15.142953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:15.144432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:15.156087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:15.157013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:15.157974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 435/830] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:15.163517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:15.280581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fa303-0658-40d4-8a2b-e8f25c990a2f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:15.335312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:15.335970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:15.336250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:15.336578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:15.337341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:15.337647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:15.339085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c656ac-10e9-43c5-8c5b-20c1b35329b0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:15.353567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 396/831] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:15.359360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:15.466664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:15.473993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ebbb5-b6a5-4ddb-b5d9-7b389ef32c8a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:15.511896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:15.512520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:15.512794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:15.513062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:15.513856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=dd1da21c00d8494587e3ede50a5a365b) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:15.514162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:15.516047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40288-3925-4698-84cf-3ee9ecf0bb67 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:15.523193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 436/832] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:15 2022] GET /identity/v3/role_assignments?user.id=dd1da21c00d8494587e3ede50a5a365b&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:15.885221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030266bb-a134-4748-abca-c2b606042d9d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:15.885742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030266bb-a134-4748-abca-c2b606042d9d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:15.885742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030266bb-a134-4748-abca-c2b606042d9d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:15.887346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 397/833] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:15.892982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:15.893260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:15.893524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:15.912458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:15.913667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:15.983277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 437/834] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:16.006933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030266bb-a134-4748-abca-c2b606042d9d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:16.074592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:16.138273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030266bb-a134-4748-abca-c2b606042d9d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:16.174388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:16.175041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.175320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.175595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.186855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:16.192350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:16.193288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:16.194131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 398/835] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:16.202516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:16.320296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:16.338726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95102b62-18b5-4f00-851f-828530a9fcb9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:16.377416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:16.378046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.378316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.378583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.379295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:16.379591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:16.380987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:16.390243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 438/836] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:16.396214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:16.516819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe6c9a-41b8-4914-ab4f-4e80eb3ab557 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:16.519332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:16.555124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:16.556104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.556535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.557001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.569069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:16.570843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7c0e4762-1170-402c-848a-d4f6d41aa777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:16.586834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:16.587994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:16.589109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:16.589109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:16.589109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:16.589109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:16.590930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 399/837] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:16.597223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:16.684414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734b96-b2d9-4c99-aa03-6b1cc9bfb6ad None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.684414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734b96-b2d9-4c99-aa03-6b1cc9bfb6ad None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.684414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734b96-b2d9-4c99-aa03-6b1cc9bfb6ad None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.684414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 400/838] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:16.690635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.691018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.691316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.708927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b4b6f-1877-4f08-b793-ed2e28a2b6c5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:16.712099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:16.713397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:16.752014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:16.752643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.752920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.753195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.753896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:16.754199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:16.755736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:16.770229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 439/839] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:16.775480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:16.796169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:16.812353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 401/840] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:16.836298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:16.912515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83176df-3964-4baf-aff8-c92964d05733 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:16.936217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:16.944832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f58840-bfc6-4ba6-8d70-ba394844f98d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:16.957969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:16.958619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.958891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.959554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.969440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:16.971562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:16.988442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:16.990762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:26:16.992998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:16.993664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:16.993664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:16.993664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:16.993664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:16.997939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 440/841] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:16.997939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.005453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:17.010893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:17.012089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:26:17.013132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:17.020300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 402/842] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:17.021660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.082206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.119598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984de9e-933c-40fa-a475-a4d8576ae3ed None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.131341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba4f9e-7813-42cf-b5c2-cb4330821b3f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.156993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.157608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.157891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.158183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.159013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:17.159344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.160994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd37c-f1e4-4360-8704-8f95eaae5560 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.169806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.170832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.171105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.171433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.172249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:17.172553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.173932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c6172-53f2-4677-983c-4212362dcd47 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.184719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 441/843] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Aug 1 16:26:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:17.184719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d68c7-2d09-4cd7-a37e-9f6b4ebb898d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.184719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d68c7-2d09-4cd7-a37e-9f6b4ebb898d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.184719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d68c7-2d09-4cd7-a37e-9f6b4ebb898d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.184719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 442/844] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:17.185722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d68c7-2d09-4cd7-a37e-9f6b4ebb898d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.189934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 403/845] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:17.196002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.196002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.196002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.212683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:17.213876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:17.216196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.285503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 404/846] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:17.286686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.313838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d68c7-2d09-4cd7-a37e-9f6b4ebb898d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.342026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.350225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.350847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.351127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.351396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.414373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aed541-06c7-45e4-8eaa-7e8f537c2b6b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.436415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.439371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.460974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.461634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.461915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.462183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.469967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.474327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.476310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:17.492750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:17.493715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:17.494634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:17.494634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:17.494634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:17.494634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:17.494634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 405/847] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:17.494908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.570155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 443/848] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:17.571451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.602234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.626631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc226-46c8-4a3c-8869-d73b27b3b761 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.682835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.684376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.684926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.685415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.687288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.690152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.699427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717512d0-ae41-4e40-a94e-ead9e8262bb2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:17.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:17.709491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:17.710306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:17.716671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 406/849] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:17.716671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.740239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.740911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.741162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.741399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.742086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:17.742384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.743750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.756094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 444/850] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:17.756922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.786196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.835234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01aba7-25c4-4903-a381-dd45e7216407 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.860079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:17.872986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce6a6-2cc4-49ce-b05d-2ba0b74e76b3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:17.873564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.874212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.875195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.875476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.892153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.893627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7be75980-8c84-46aa-ad05-7c684aa5ba49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:17.908694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:17.909651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:17.910617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:17.910617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:17.910617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:17.910617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:17.910847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:17.910847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.910847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.911063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 407/851] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:17.911130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.911130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:17.911130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:17.911313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084ac28-9d7f-49b1-bb05-f6ee8208a02e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:17.911313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084ac28-9d7f-49b1-bb05-f6ee8208a02e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:17.911944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084ac28-9d7f-49b1-bb05-f6ee8208a02e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:17.912318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:17.912961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 408/852] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:17.913482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084ac28-9d7f-49b1-bb05-f6ee8208a02e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:17.924616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 445/853] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:17.924616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.027269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084ac28-9d7f-49b1-bb05-f6ee8208a02e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.030077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfaadf7-f0eb-4c7f-9e72-65ba435a01a6 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.034596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.063827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.064426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.064540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.064798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.065064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.065166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.065338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.065605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.065854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:18.066147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.067627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89706bb9-9e39-42a6-b333-29dbdf0fcb60 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.073482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.074908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.080360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 446/854] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:18.081879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.082195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.082463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:18.086498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:18.087434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 409/855] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:26:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:18.088409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.102291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:18.103561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:18.111242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.171291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 447/856] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:18.172182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.200143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f405a4-0c78-4deb-adcf-ac0c46ace7f3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.236174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.237060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.237335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.237597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.238273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:18.238572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.239977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.244378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 410/857] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:18.245148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.280099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.291853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cfefd-f465-4612-82f8-f3b7ee3ab103 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.331632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.332340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.332656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.332930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.340830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.342357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:18.355300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:18.357188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:18.359018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:18.361761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 448/858] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:18.362928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.366476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.371042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1167d-66ff-4343-8fb7-a32a2eb2469c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.415172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.416134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.416134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.416416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.417938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:18.417938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.419188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.440161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 411/859] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:18.441015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.488673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8b3d9-51bc-441c-9a54-d2507e6d6ec8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.533487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.534427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.534802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.535166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.536038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.537872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.539683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:18.550490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:18.551476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:18.552383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 449/860] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:18.553214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.557412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f805d-ca5f-4238-b64f-b4ab162e0eeb None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.600884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.601793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.602219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.602641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.603651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:18.604142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.605585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.623999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 412/861] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:18.624993 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.631130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.675811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e2642-cc56-4e12-92cd-8aefb396e320 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.719427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.720310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.720653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.721031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.753831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea287741-d826-469c-8e2b-6018e31759d9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.792672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.793233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.793507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.793780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.794466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:18.794771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.796351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.803647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.805145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:18.806457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 413/862] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:18.807411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.824947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.886664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:18.928874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f44d2-d2b0-40c5-a42f-a53e746a37a5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:18.941370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 450/863] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:18.942295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:18.986490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:18.987531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:18.988054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:18.988529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:18.989957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:18.990511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:18.993197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.015813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 414/864] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/role_assignments?user.id=4177b0f6e8204e5285ee4fdaeb2aa4c9&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:19.015813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:19.083112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97dbfb-daa6-4e89-a7fc-e5d61607f051 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:19.116331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:19.139805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:19.140776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:19.141168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:19.141781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:19.142522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:19.142833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:19.145356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.147459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec379641-d235-461c-9ef4-3720ce146677 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:19.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 451/865] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:19.202095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:19.203183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:19.203685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:19.204162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:19.219718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:19.221507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:19.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:19.242942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:19.244136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:19.244136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:19.244136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:19.244136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:19.244136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 415/866] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:19.250923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:19.293460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:19.379072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac63d63-c6b8-4252-b76d-32bcaaa392a8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:19.418680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:19.419304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:19.419574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:19.419880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:19.420580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:19.420877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:19.422470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.434868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 452/867] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:19.440718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:19.554559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a18d22-5d11-4c67-99de-c67970f5ee2c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:19.582707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:19.592352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:19.592994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:19.593328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:19.593597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:19.601216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:19.602669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:19.614826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:19.615825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:19.616750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 416/868] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:19.625271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:19.730280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:19.742480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375fb4b-f432-4fa0-80a7-e481d1e6f29f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:19.778853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:19.779501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:19.779822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:19.780095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:19.781243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:19.781548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:19.783053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573dd5-48e7-42e0-bd69-9c07460cbacd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.799873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 453/869] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:19.802490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:19.914684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f85f3-42b2-4b66-8d57-4527971ff41d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:19.952051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:19.952624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:19.952901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:19.953169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:19.954330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=5a83ef8e97ae4500943dfbcdd3fb7587) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:19.955207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:19.956989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23878b1c-d4bd-47f3-8db8-3c42aad1eb75 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:19.964534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 417/870] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:19 2022] GET /identity/v3/role_assignments?user.id=5a83ef8e97ae4500943dfbcdd3fb7587&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:20.329769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be4f67-d902-4f9b-a20f-c2a2cafc72da None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:20.330176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be4f67-d902-4f9b-a20f-c2a2cafc72da None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:20.330370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be4f67-d902-4f9b-a20f-c2a2cafc72da None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:20.331864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 454/871] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:20.338012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b62e51-90a8-4199-a6d7-efc8816da392 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:20.338988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b62e51-90a8-4199-a6d7-efc8816da392 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:20.339492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b62e51-90a8-4199-a6d7-efc8816da392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:20.360384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-64b62e51-90a8-4199-a6d7-efc8816da392 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:20.362151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-64b62e51-90a8-4199-a6d7-efc8816da392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:20.415827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:20.441609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 418/872] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:20.463642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be4f67-d902-4f9b-a20f-c2a2cafc72da None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:20.572931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be4f67-d902-4f9b-a20f-c2a2cafc72da None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:20.608752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d650e1-736c-4809-a7b2-32ac177df4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:20.609692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d650e1-736c-4809-a7b2-32ac177df4be None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:20.609997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d650e1-736c-4809-a7b2-32ac177df4be None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:20.610272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d650e1-736c-4809-a7b2-32ac177df4be None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:20.610968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d650e1-736c-4809-a7b2-32ac177df4be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:20.612372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d650e1-736c-4809-a7b2-32ac177df4be None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:20.629676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 455/873] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:20 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:21.270704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd64c5-fee1-4674-8e73-cb2d1b3a8a1f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.271139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd64c5-fee1-4674-8e73-cb2d1b3a8a1f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.271248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd64c5-fee1-4674-8e73-cb2d1b3a8a1f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.272906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 419/874] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:21.278221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b00d5-aed2-4e60-a55a-d50e43e5d24e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.278504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b00d5-aed2-4e60-a55a-d50e43e5d24e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.278764 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b00d5-aed2-4e60-a55a-d50e43e5d24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.298029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9a6b00d5-aed2-4e60-a55a-d50e43e5d24e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:21.299206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9a6b00d5-aed2-4e60-a55a-d50e43e5d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:21.302200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:21.383238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 456/875] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:21.404712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd64c5-fee1-4674-8e73-cb2d1b3a8a1f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:21.438911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9dbba-5e59-4d5d-9a1c-de7ec942681b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.439249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9dbba-5e59-4d5d-9a1c-de7ec942681b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.439453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9dbba-5e59-4d5d-9a1c-de7ec942681b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.441867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 457/876] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:21.449154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.449714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.450187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.469846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:21.472017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:21.522551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd64c5-fee1-4674-8e73-cb2d1b3a8a1f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:21.551849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 458/877] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:21.559492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:21.560171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.560446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.560715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.561339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:21.562764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:21.572100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:21.573313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:21.574297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 420/878] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:21.576929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:21.579171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:21.641019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:21.650536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:21.690136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c81e6f-709a-418d-a503-5020ce8c1f3e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:21.728326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:21.729316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.729897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.730394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.731299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:21.732956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:21.739113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf38ee-0cd5-4df6-8549-f8d89bd1ca13 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:21.759918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 459/879] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:21 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:21.806802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:21.807353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.807625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.807938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.808638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:21.808940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:21.810396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51576-3edf-41e3-9c5b-fd6efccbbeca None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:21.816972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 421/880] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:21.823040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:21.939047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab5e45-9c5f-4b2f-b8d4-9bba16668fa7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:21.969324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:21.980547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:21.981181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:21.981454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:21.981721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:21.982402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:21.982701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:21.984255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a40ba-b602-4909-89af-97d96b200275 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:21.989517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 460/881] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:22.153541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc16085-17a2-414c-a49f-512a2065dbc7 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.154050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc16085-17a2-414c-a49f-512a2065dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.154476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc16085-17a2-414c-a49f-512a2065dbc7 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.156508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 422/882] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:22.162288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.162775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.163188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.182655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:22.184091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:22.259842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 461/883] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:22.283617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc16085-17a2-414c-a49f-512a2065dbc7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:22.394216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc16085-17a2-414c-a49f-512a2065dbc7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:22.402081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:22.430930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:22.432258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.432690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.433107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.444064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:22.449180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:22.450310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:22.451385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:22.453902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 423/884] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 16:26:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:22.460004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:22.569872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b2c7-a989-46a9-8c93-ec7f5f368bb8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:22.591886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:22.607185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:22.607847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.608124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.608399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.609695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:22.610015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:22.611405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059b08f-9012-4129-b47e-96397eba0872 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:22.620599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 462/885] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:26:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:22.626977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:22.737737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9103a-9068-4ae9-af7b-d5f511d5421b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.737935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9103a-9068-4ae9-af7b-d5f511d5421b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.738189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9103a-9068-4ae9-af7b-d5f511d5421b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.740515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 463/886] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:22.742146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d396-2b1b-4b5b-887a-8703c86e0e8a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:22.749303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.749303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.749303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.768266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:22.769470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:22.792520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:22.794122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:22.794601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:22.795344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:22.812785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:22.814796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:22.835698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:22.836718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:22.837680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:22.837680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:22.837680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:22.837680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:22.839958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 424/887] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:22.851873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:22.867846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 464/888] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:22.881364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:22.901982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:22.994953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176880be-b9fc-4940-9c45-870ca6c30b14 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.013750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a879-6f1d-4392-ac1b-ea49650fd8e8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.029169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:23.046731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.047660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.048445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.048445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.049492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:23.049890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.052057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.057941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.058961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.059425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.059895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.060693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.062315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.075431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 425/889] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:23.075633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:23.076609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:23.077502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 465/890] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:23.083927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.085691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.185180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:23.215366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e0097-fc2d-43eb-af04-f90790805e71 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.219648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aaae9-fa99-43c5-98d1-a02d67337a78 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.253372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.253908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.254181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.254447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.255121 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:23.255417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.256824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.269119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 466/891] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:23.270507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.289098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.290137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.290650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.291396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.301855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.304486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:23.321320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:23.322549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:23.323814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:23.325280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:23.333194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 426/892] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:23.334010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.410660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ed42-ce02-4010-9bb0-dadc76e24791 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.438266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54377565-5d3a-4e49-a9c0-be163d047297 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.458555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.460047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.460812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.461313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.462469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:23.463368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.465567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.478420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 467/893] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:23.480470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.480902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.481155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.481516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.481790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.482744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:23.483057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.484697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.495415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:23.514122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 427/894] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:23.516321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.613263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b75c2c-0a40-4fe9-a107-3b23de5ff39a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.617179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:23.656631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fdba8-f260-4aca-b84d-b733504bee34 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.663381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.664044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.664451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.664727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.665499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] RBAC: Adding query filter params (interface=public, service_id=80038a09e33e482da331d4256b006865, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:23.665813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.667446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.678073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 468/895] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/endpoints?service_id=80038a09e33e482da331d4256b006865&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:23.679294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.708007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c55367-56bb-473f-b6bb-d81eb9e014e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.709092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c55367-56bb-473f-b6bb-d81eb9e014e5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.709585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c55367-56bb-473f-b6bb-d81eb9e014e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.710057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c55367-56bb-473f-b6bb-d81eb9e014e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.789484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:23.797407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394bb70-ad47-469f-8aeb-f709db6324de None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:23.799858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c55367-56bb-473f-b6bb-d81eb9e014e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.801693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c55367-56bb-473f-b6bb-d81eb9e014e5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.836426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:23.837269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:23.837494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:23.837768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:23.838475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:23.839991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:23.852229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 469/896] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:26:23.928617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 428/897] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:23.934894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:23.974808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.054639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44ca0b-de4d-469d-9a34-5c5102760b42 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.068460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c038e6-ac56-4ace-93aa-61b383444e25 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.069081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c038e6-ac56-4ace-93aa-61b383444e25 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.069607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c038e6-ac56-4ace-93aa-61b383444e25 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.071488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 429/898] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:24.077441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c37e9-9f0a-41fa-a695-ebc7ef10a455 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.078198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c37e9-9f0a-41fa-a695-ebc7ef10a455 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.078743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c37e9-9f0a-41fa-a695-ebc7ef10a455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.098282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-da7c37e9-9f0a-41fa-a695-ebc7ef10a455 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:24.100079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-da7c37e9-9f0a-41fa-a695-ebc7ef10a455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:24.104444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.105042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.105324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.105599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.125572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.127257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:24.145321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:24.146301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:24.147194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:24.147194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:24.147194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:24.147194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:24.147194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 470/899] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:24.153017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.184705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 430/900] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:24.186988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1ce8e-08e9-421f-8f8f-07b9a51ff2a5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.187491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1ce8e-08e9-421f-8f8f-07b9a51ff2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.188359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1ce8e-08e9-421f-8f8f-07b9a51ff2a5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.190614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 431/901] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:24.200240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.200510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.200797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.221572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:24.222764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:24.247437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.276565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.296787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8091bb-d543-4e53-bdab-cd43c1c801d7 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.307890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 432/902] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:24.307890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.349104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.349984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.350407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.350848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.351686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:24.352191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.353732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.367499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 471/903] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:24.368384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.442938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51d34-974e-4d8a-9e2c-51f7c95f8b25 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.479885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19bee6-e747-4a41-9e66-e0842c192950 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.484149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.484874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.485141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.485372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.486039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.487376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:24.497434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:24.498458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:24.499999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 433/904] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:24.501118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.525224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.525852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.526128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.526400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.527097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.527552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.528584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.547826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 472/905] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:24 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:24.547826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.551893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.655749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61b62-1ec5-44e4-b9cd-1191399d4293 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.724732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.725387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.725719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.725991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.726577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8f6d-ce3d-4e24-8d66-c6c3e2898500 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.734555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.735358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:24.749586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:24.750577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:24.751464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 434/906] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:24.757237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.780404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.781806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.782089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.782403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.783097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:24.783404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.784847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.790929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 473/907] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:24.800505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.865165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.876133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:24.897827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8e12c-3c41-4097-861e-2bca5eab02e8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.909541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d25b1-8c55-4341-932f-6aa9806429fb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:24.938766 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.939879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.940299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.940668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.941588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:24.942000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.943565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.955307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:24.956496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:24.956944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:24.957380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:24.958209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:24.958690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:24.960352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:24.963572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 435/908] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:24.964663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:24.972168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 474/909] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:24.973372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:25.101068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3a3d8-bd73-482f-b185-778d9a676f43 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:25.119221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea6a8-d315-4eb7-bbf2-95014fe2f83b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:25.149429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:25.150353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:25.150705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:25.151111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:25.152021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] RBAC: Adding query filter params (role.id=0fdcdf07d33844aebfc4c6168ec857ed, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=5a83ef8e97ae4500943dfbcdd3fb7587) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:25.152400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:25.154292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:25.157701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:25.159269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:25.165151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:25.166345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:25.166975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:25.167414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:25.168325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 436/910] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/role_assignments?user.id=5a83ef8e97ae4500943dfbcdd3fb7587&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=0fdcdf07d33844aebfc4c6168ec857ed => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:25.168618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] RBAC: Adding query filter params (interface=public, service_id=42e15c91404f4917b3245b4f02bedab2, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:25.169509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:25.171356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbb0ac-698b-4404-aacb-1c0b55e4d8c6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:25.178163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 475/911] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:24 2022] GET /identity/v3/endpoints?service_id=42e15c91404f4917b3245b4f02bedab2&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:25.184164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:25.309054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ad137-75bb-4020-b144-b2348e92df56 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:25.349790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d91db-ae7a-495f-be2a-c7fb0d1d5b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:25.350772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d91db-ae7a-495f-be2a-c7fb0d1d5b53 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:25.351227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d91db-ae7a-495f-be2a-c7fb0d1d5b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:25.351670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d91db-ae7a-495f-be2a-c7fb0d1d5b53 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:25.352525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d91db-ae7a-495f-be2a-c7fb0d1d5b53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:25.354074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d91db-ae7a-495f-be2a-c7fb0d1d5b53 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:25.361755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 437/912] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:25 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:26:25.969796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2f51a-4dad-45e0-98c0-daa8c5515a2c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:25.970399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2f51a-4dad-45e0-98c0-daa8c5515a2c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:25.970399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2f51a-4dad-45e0-98c0-daa8c5515a2c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:25.971872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 476/913] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:25.977577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:25.978004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:25.978174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:25.997046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:25.998592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:26.067155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:26.080113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 438/914] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:26.105113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2f51a-4dad-45e0-98c0-daa8c5515a2c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:26.216742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2f51a-4dad-45e0-98c0-daa8c5515a2c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:26.255345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:26.255946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:26.256220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:26.256491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:26.257119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:26.258496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:26.268437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:26.269480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:26.270336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 477/915] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:26.275823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:26.386174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c3e5-a9a7-4539-9fb8-b245b036e561 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:26.424773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:26.425775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:26.426247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:26.426739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:26.428093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:26.428505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:26.429924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:26.435815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 439/916] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:26.448040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:26.515858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:26.563171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fa31-f37f-41fa-b03a-6681ab9a4fd4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:26.602196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:26.603050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:26.603412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:26.603771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:26.604664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:26.605054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:26.607029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2d24f-6e68-45c6-8c7b-0190a0542f32 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:26.613901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 478/917] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Aug 1 16:26:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:26.619310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:26.688509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:26.735595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ab73-5356-45f4-9293-3bbb948e7599 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:26.777910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0053bb5-efa8-48fc-b5ec-075e586a79b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:26.778469 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053bb5-efa8-48fc-b5ec-075e586a79b9 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:26.778785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053bb5-efa8-48fc-b5ec-075e586a79b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:26.779098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053bb5-efa8-48fc-b5ec-075e586a79b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:26.779936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053bb5-efa8-48fc-b5ec-075e586a79b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:26.781880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053bb5-efa8-48fc-b5ec-075e586a79b9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:26.804268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 440/918] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:26.855347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540c9f6-76d0-47c0-8dd2-f107233f67cd None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:26.855663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540c9f6-76d0-47c0-8dd2-f107233f67cd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:26.855935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540c9f6-76d0-47c0-8dd2-f107233f67cd None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:26.857709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 479/919] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:26.864883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:26.865250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:26.865447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:26.887432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:26.900897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:26.983788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 441/920] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:27.006187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540c9f6-76d0-47c0-8dd2-f107233f67cd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.135201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540c9f6-76d0-47c0-8dd2-f107233f67cd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.169001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:27.179793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.179793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.179793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.179793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.180201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:27.183787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-da8e0ea7-adba-4e87-8425-b7460b7974b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:27.195569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:27.196662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:27.197506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 480/921] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:27.201316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.249680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:27.313482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2531ae-114f-44d8-bf52-9b05368cc4ab None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.353403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f94d-ee7c-4212-94df-36469651587f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.353403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f94d-ee7c-4212-94df-36469651587f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.353403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f94d-ee7c-4212-94df-36469651587f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.354234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 481/922] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:27.358132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.358132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.358132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.358132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.358835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:27.359214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:27.359919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729577c4-7195-4c13-8e3e-c6a9bb6f2775 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.360205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729577c4-7195-4c13-8e3e-c6a9bb6f2775 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.360483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729577c4-7195-4c13-8e3e-c6a9bb6f2775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.361427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:27.367377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 442/923] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:27.373285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.382824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-729577c4-7195-4c13-8e3e-c6a9bb6f2775 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:27.384115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-729577c4-7195-4c13-8e3e-c6a9bb6f2775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:27.479991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 482/924] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:27.491468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb156b-dd9e-4e68-b6c9-6730bccfad49 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.492716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb156b-dd9e-4e68-b6c9-6730bccfad49 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.492716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb156b-dd9e-4e68-b6c9-6730bccfad49 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.493871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 483/925] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:27.499724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.500014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.500285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.507977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c12d4-53f1-4c56-a6ea-0129a68cc102 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.511434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:27.519271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:27.520700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:27.551446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.552340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.552660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.552955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.553880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:27.554257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:27.556080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:27.562613 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 443/926] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:27.564978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.603811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 484/927] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:27.603811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.655971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:27.688243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325e10b-b0bd-49e1-9328-e3b6978cd6cd None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.724874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.725526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.725799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.726076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.728277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff61499-69aa-4b98-8d0c-66bbf187544e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.737576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:27.742414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:27.743324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:27.744231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 444/928] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:27.744927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.770695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.771986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.772408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.772675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.773425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] RBAC: Adding query filter params (interface=public, service_id=0e0db80fa9444e419e4a9b72f17d7a4a, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:27.773727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:27.774566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:27.775090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:27.780741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 485/929] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/endpoints?service_id=0e0db80fa9444e419e4a9b72f17d7a4a&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:27.785968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.866516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90617-f866-45b1-ad46-be26f5a0b09b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.907242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.907942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.908219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.908516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.909156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:27.910533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:27.916202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099a35-bbe1-4c4c-813a-2529a7fffcad None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:27.920232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:27.921258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:27.922095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 445/930] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:27.922965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:27.936085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:27.960192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:27.960812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:27.961083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:27.961347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:27.962095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:27.962391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:27.963747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:27.972566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 486/931] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:27.973286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.026540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:28.036657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87595678-3fa8-40ce-a42b-9e141f6f9c2c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.075779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.076424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.076703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.076977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.078001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.079438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.095803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 446/932] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:26:28.095803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.096185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0e962-4726-4062-8535-ae8fd803d0b1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.143746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.145285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.145806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.146084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.147243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:28.147549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.149567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.157837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 487/933] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:28.163713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.227923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778f5a4-10a1-436a-ad84-f196d6236301 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.229305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:28.277720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.278728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.279210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.279680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.295602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.297274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:28.315388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:28.316441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:28.317347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:28.317347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:28.317347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:28.317347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:28.317607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2448f4-2123-4359-ae54-2e8a27e5e552 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.317684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 447/934] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:28.322751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.334682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:28.375349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.376425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.376928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.377394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.378405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:28.378906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.380536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.386980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 488/935] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:28.393978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.426326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f77d7-61c4-4a09-8dcb-3e375f754c35 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.461366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.462355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.462809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.463257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.464465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:28.465020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.467410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.481440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 448/936] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:28.486968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.511375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f217426-5fbe-49e8-8427-17368a5c6d5d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.513586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:28.556268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:28.558434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.560500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.560992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.561652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.563048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.565077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.600132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf53915-a68f-492d-b1fd-9a1f0bb575f3 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.600534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 489/937] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:26:28.642061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.642723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.642999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.643271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.651526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.653065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:28.666624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:28.667648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:28.668678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:28.679446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 449/938] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:26:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:28.688806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.816645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79da6e-d9b4-404b-8f12-c3fbb3354729 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:28.854233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:28.854869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.855144 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.855414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:28.856268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:28.856576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:28.858033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a67e5-5bb0-4498-b6b2-5be2472c77b1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:28.881013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 490/939] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 16:26:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:28.887202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:28.999081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b653f5-1a24-49a6-beda-03e3e02406b8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:28.999487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b653f5-1a24-49a6-beda-03e3e02406b8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:28.999620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b653f5-1a24-49a6-beda-03e3e02406b8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.001331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 491/940] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:29.006610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.006890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.006990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463b129-1c6b-41d0-af0f-22a9d2896734 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.007188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.025682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:29.026884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:29.031824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:29.040429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:29.049911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.050829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.051373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.051818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.068908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.070612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:29.087492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:29.088677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:29.089808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:29.089808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:29.089808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:29.089808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:29.091824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 450/941] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:29.097475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.107662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 492/942] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:29.130921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.220100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3be097-4c22-4ce1-bdd4-a956836861b2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.232638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5198e-3017-4d05-bf35-6470557ec9e3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.260051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.261177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.261659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.262132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.263138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:29.263737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.266172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.268686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.269410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.269682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.269949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.270694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.272099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.274738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING keystone.server.flask.common [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 16:26:29.282854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 451/943] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:29.292836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.300059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.sql.core [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 16:26:29.300059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 16:26:29.300059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [parameters: {'id': 'c15c7e9c5ced4c2980b4b7ba7a7b81cb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 16:26:29.300059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80349) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c15c7e9c5ced4c2980b4b7ba7a7b81cb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c15c7e9c5ced4c2980b4b7ba7a7b81cb. Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:26:29.320753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result.read() Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 16:26:29.321659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application yield resource Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise self.value Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application session.commit() Aug 01 16:26:29.322696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.session.flush() Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application compat.raise_( Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 16:26:29.323612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 16:26:29.324577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application result.read() Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:26:29.325390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [parameters: {'id': 'c15c7e9c5ced4c2980b4b7ba7a7b81cb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.326259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:29.327151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c15c7e9c5ced4c2980b4b7ba7a7b81cb. Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 493/944] 10.222.0.45 () {60 vars in 1140 bytes} [Mon Aug 1 16:26:29 2022] POST /identity/v3/projects => generated 262 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:29.331155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.334083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:29.348202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:29.429880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576c06-b293-4086-977f-90c1599aea81 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.457032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6c9af-5087-42a6-b5e4-cd51ea251fef None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.468503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.469516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.470002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.470530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.478211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.479995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:29.492759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:29.493697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:29.494582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:29.497592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 452/945] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:29.506567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.520755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.521417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.521696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.521968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.530882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.532711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:29.547525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:29.549224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:29.550927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:29.553770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 494/946] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:29.554697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.620787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:29.646528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:29.653447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e954a-7f16-42bf-a8b5-15ae6fd89b70 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.667248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373ee8-1b23-45ef-bf75-0573edba4c6a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.699391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.700557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.700998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.701792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.703406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:29.703896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.706591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.710356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.711101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.711490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.711886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.712706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:29.713104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.714628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d0405-1424-4237-8413-10042ba482c2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.722665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 453/947] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:29.734401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:29.740883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 495/948] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:29.858277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e6880-d323-43da-9b80-cc0af153d7d4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:29.899883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14eb43-2c04-493c-a99f-58f2edbb3c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:29.900560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14eb43-2c04-493c-a99f-58f2edbb3c95 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:29.900833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14eb43-2c04-493c-a99f-58f2edbb3c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:29.901152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14eb43-2c04-493c-a99f-58f2edbb3c95 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/5a83ef8e97ae4500943dfbcdd3fb7587/roles/0fdcdf07d33844aebfc4c6168ec857ed` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:29.903635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14eb43-2c04-493c-a99f-58f2edbb3c95 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=5a83ef8e97ae4500943dfbcdd3fb7587, role_id=0fdcdf07d33844aebfc4c6168ec857ed)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:29.905232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14eb43-2c04-493c-a99f-58f2edbb3c95 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:29.924058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 454/949] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:26:29 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/5a83ef8e97ae4500943dfbcdd3fb7587/roles/0fdcdf07d33844aebfc4c6168ec857ed => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:26:30.297541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28828674-f842-48d3-8a39-7f53021c6df5 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.298111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28828674-f842-48d3-8a39-7f53021c6df5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.298111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28828674-f842-48d3-8a39-7f53021c6df5 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.299754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 496/950] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:30.305324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.305806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.305912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.326742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:30.328495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:30.343389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:30.409558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 455/951] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:30.433054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28828674-f842-48d3-8a39-7f53021c6df5 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:30.543358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28828674-f842-48d3-8a39-7f53021c6df5 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:30.567488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:30.580140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:30.580794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.581063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.581328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.582010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:30.583397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:30.593222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:30.594267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:30.595114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 497/952] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:30.601229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:30.737127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa715bca-bf36-44fe-8710-26d9650bcb39 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:30.774081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:30.774960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.775405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.775864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.776692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:30.777144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:30.778658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216cdc6-9537-4680-b104-85547ab7d681 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:30.784466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 456/953] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:30.790518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:30.876708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01da9-73d7-4178-a05c-ce8193f76cd3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.876907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01da9-73d7-4178-a05c-ce8193f76cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.877276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01da9-73d7-4178-a05c-ce8193f76cd3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.878945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 457/954] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:30.883901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.884252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.884676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.898103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fce8a-b93f-4a50-928a-7b6210950fb1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:30.906987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:30.908380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:30.912865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:30.939667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:30.940326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:30.940606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:30.940879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:30.941557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:30.941859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:30.943309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:30.953367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 498/955] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:30.955244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:31.000288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 458/956] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:31.024137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:31.031689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:31.101068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171609a-3b65-42ef-a2a8-4503c3290a9b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:31.139505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:31.140239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:31.140542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:31.140941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:31.141737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:31.143207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:31.150945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a26010-9390-4218-b53f-976d8552e863 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:31.175803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 499/957] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:31.199010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:31.200070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:31.200552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:31.200987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:31.201919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:31.203734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:31.214557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:31.215732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:31.216715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:31.219755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 459/958] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:31.220165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:31.225680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:31.362900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e0c01-af95-44bb-9039-c2e57c79e883 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:31.380162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:31.421053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:31.421773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:31.422072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:31.422442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:31.423296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:31.423675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:31.426033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80938ab1-0b91-497a-a044-1676d9b21a48 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:31.437270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 500/959] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:31.444249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:31.670180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aafc6-332b-43cc-a0b5-2b51f4cc4e06 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:31.762848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:31.763512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:31.766018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:31.766294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:31.767012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:31.767314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:31.768736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8729-67be-4699-aca4-06a6731ee052 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:31.786312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 460/960] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:32.087323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f916-c9a5-42e5-9c83-85489790520f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.087581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f916-c9a5-42e5-9c83-85489790520f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.087887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701f916-c9a5-42e5-9c83-85489790520f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.089628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 501/961] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:32.095802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.096095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.096335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.115898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:32.117155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:32.152940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cef73-6f7a-4e66-b496-f6a5a42b974d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.153263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cef73-6f7a-4e66-b496-f6a5a42b974d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.153493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cef73-6f7a-4e66-b496-f6a5a42b974d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.154928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 502/962] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:32.160844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.161438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.161933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.181427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:32.183333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:32.191074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 461/963] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:32.192082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:32.218158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.262449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 503/964] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:32.285543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.322779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c5460-10de-4db9-81eb-0eb5baa5ebb7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:32.360720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:32.361474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.361973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.362470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.363850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:32.365536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:32.368292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING keystone.server.flask.common [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 16:26:32.381809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.sql.core [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 16:26:32.381809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 16:26:32.381809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [parameters: {'user_id': 'f4f912b73ef74d12b3ab6c4fdc91f565', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 16:26:32.381809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80348) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 16:26:32.389650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:26:32.390566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application yield resource Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise self.value Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application session.commit() Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 16:26:32.391495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.session.flush() Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application compat.raise_( Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 16:26:32.393170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:26:32.395234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application result.read() Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:26:32.397259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f4f912b73ef74d12b3ab6c4fdc91f565', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:32.399200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 16:26:32.401123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 462/965] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Aug 1 16:26:32 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:32.404560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.409394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:32.412261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3773253-354e-45ae-8d56-1af80f121d88 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:32.459955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:32.461460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.462150 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.462615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.478072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:32.480128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:32.485981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:32.487000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:32.488082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:32.495025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 504/966] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 16:26:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:32.497621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.542639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fa010-9922-4c42-963a-c2a0d47eaebf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:32.582239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:32.582991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.583349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.583926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.591781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:32.593668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:32.606652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:32.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:26:32.608918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:32.612111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 463/967] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:32.614075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.615133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622729-e3dc-4d37-9cfc-f4254e4bbe94 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:32.651436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:32.652578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.653047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.653531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.654460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:32.655003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:32.656701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:32.670230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 505/968] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:26:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:32.671656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.699807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:32.724673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe352-8e4a-4829-8a97-9b3e906de186 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:32.742086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:32.763720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:32.764446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.764735 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.765003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.765843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:32.766190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:32.767645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:32.783954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5d04-2546-44c8-b88a-532b0fbb851d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:32.812179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 464/969] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Aug 1 16:26:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:32.820404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:32.821472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:32.821967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:32.822445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:32.836034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:32.837903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:32.855153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:32.856269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:32.857167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:32.857167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:32.857167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:32.857167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:32.863936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 506/970] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:32.866254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:32.993008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc0750-6abc-45b7-a55d-994d3d272254 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.032851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.033550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.033845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.034104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.034879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:33.035185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.036615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.050765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 465/971] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:33.057019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.149626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:33.177217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f8ea7-2367-4fce-9ee7-a703be3b7931 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.211345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.212011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.212287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.212574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.221248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.222701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:33.234785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:33.240981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:33.241976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 507/972] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:33.242603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.279360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:33.361925 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245a7c2-e2b9-47ef-b2c6-69625d8e2da6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.396465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.397253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.397527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.397796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.398591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:33.398892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.400414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1085a-0390-4ae5-8387-41280da015e9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.428505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 466/973] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:33.430199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.482595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d6314-dd44-4b4a-af03-53b15f2b2cba None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.483187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d6314-dd44-4b4a-af03-53b15f2b2cba None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.483187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d6314-dd44-4b4a-af03-53b15f2b2cba None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.484706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 467/974] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:33.489900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.490182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.490451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.509688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:33.511202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:33.562482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fdd15-499d-4e4b-9c03-7c47ab8e31f2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.584759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 468/975] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:33.599688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.600341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.600619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.600891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.603697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:33.606888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.632687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:33.684435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.685863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.733904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8c11c-bb89-4715-8c4d-5898bc54efdb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.774824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.775881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.776342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.776805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.777875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.780515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:33.796332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:33.797431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:33.798286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:33.799050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 469/976] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:33.819665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 508/977] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:33.819914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.821429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.857899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:33.912324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:33.930839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71845fa2-b7e6-4d4a-a3c7-91e6fcd5e804 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.935511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996d8e-07b2-48a3-a44a-276066583575 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:33.967029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.967892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.968180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.968445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.969138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:33.969354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:33.969502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.969995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.970267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.970533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.970939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc843d66-26dc-4ea4-a541-07b510ef17b3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.975815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 470/978] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:33.977183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a444ba-fc0c-480a-a73c-a15c5df0a48e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:33.977307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a444ba-fc0c-480a-a73c-a15c5df0a48e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:33.977503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a444ba-fc0c-480a-a73c-a15c5df0a48e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:33.978862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 471/979] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:33.981318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:33.982757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:33.983534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a444ba-fc0c-480a-a73c-a15c5df0a48e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-cb120ad1-35d6-4de4-95ca-0487d1687973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:33.997927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:33.998942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:33.999870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:33.999870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:33.999870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:33.999870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:33.999870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 509/980] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:34.005330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.005606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.005873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.023831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:34.024882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:34.094753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 510/981] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:34.095608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.104549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a444ba-fc0c-480a-a73c-a15c5df0a48e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.117006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:34.148308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.148932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.149208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.149483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.150168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:34.150470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.151897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.158314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 472/982] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:34.159026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.189730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:34.218345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb169-ee89-4a08-b986-c5da02e2d366 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.253396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.254022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.254299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.254570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.255258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:34.255651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.257034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.269324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 511/983] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:34.270133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.281487 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc63ac-249b-4686-881f-d50fd52836f8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.317298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.317925 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.318204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.318477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.319172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.320588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.338202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 473/984] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:34 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:34.343152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.348387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:34.384660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39414de-2cd0-459d-898c-eb6b7e5861f2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.430267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.431006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.431313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.431414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.432208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] RBAC: Adding query filter params (interface=admin, service_id=0e0db80fa9444e419e4a9b72f17d7a4a, region_id=RegionOne) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:34.432523 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.433926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.442065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 512/985] 10.222.0.45 () {56 vars in 1249 bytes} [Mon Aug 1 16:26:34 2022] GET /identity/v3/endpoints?service_id=0e0db80fa9444e419e4a9b72f17d7a4a&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:34.447337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:34.449212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.488906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e1e8-832d-4f94-a129-497e416fdd50 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.537674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.538336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.538614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.538897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.547168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.548665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:34.560666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:34.561713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:34.562602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 474/986] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:34.568635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.583727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6325a0-ea61-441e-9a5a-ce47225762cd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.620537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.621506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.622112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.622384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.623361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.624842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.627140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:34.639806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 513/987] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:34 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:26:34.701366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaac7a2-4f0b-46a3-84a1-0f5d405f9e4e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.747021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.747691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.748002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.748270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.749236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:34.749539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.751028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d59b5-b772-44d3-b120-894e8785ca8b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.752993 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:34.768172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 475/988] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:34.772951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:34.894078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c540a-160a-42dc-ba20-34b4492d9261 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:34.928226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:34.928849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:34.929122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:34.929486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:34.930192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] RBAC: Adding query filter params (role.id=0fdcdf07d33844aebfc4c6168ec857ed, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=5a83ef8e97ae4500943dfbcdd3fb7587) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:34.930498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:34.931946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56609fa0-e70f-480d-9047-3484fbe005a0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:34.938247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 514/989] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:34 2022] GET /identity/v3/role_assignments?user.id=5a83ef8e97ae4500943dfbcdd3fb7587&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=0fdcdf07d33844aebfc4c6168ec857ed => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:36.464159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351a96-7b41-40fc-90c4-af5a9ea89160 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:36.464700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351a96-7b41-40fc-90c4-af5a9ea89160 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:36.464700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351a96-7b41-40fc-90c4-af5a9ea89160 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:36.466075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 476/990] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:36.472363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e75ee-d601-4ff7-aef8-24d63206e7c8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:36.472830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e75ee-d601-4ff7-aef8-24d63206e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:36.472930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e75ee-d601-4ff7-aef8-24d63206e7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:36.491743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-070e75ee-d601-4ff7-aef8-24d63206e7c8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:36.492956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-070e75ee-d601-4ff7-aef8-24d63206e7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:36.553378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:36.562933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 515/991] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:36.585136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351a96-7b41-40fc-90c4-af5a9ea89160 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:36.706845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351a96-7b41-40fc-90c4-af5a9ea89160 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:36.711529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503af3-db23-4f66-9a3f-15b5d9cae4e2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:36.712002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503af3-db23-4f66-9a3f-15b5d9cae4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:36.712138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0503af3-db23-4f66-9a3f-15b5d9cae4e2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:36.713805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 516/992] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:36.719674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:36.719969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:36.720202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:36.750183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:36.750429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:36.751069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:36.751347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:36.751618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:36.752281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:36.752615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:36.753664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:36.763275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:36.764604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:36.765801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:36.767927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 477/993] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:36.770513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:36.831863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:36.833956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 517/994] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:36.856219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:36.880195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bda005-6d6a-4512-b0b0-f60c7a57b380 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:36.916936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:36.917740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:36.918026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:36.918297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:36.918983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:36.919286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:36.920733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:36.929097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 478/995] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:36.934287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:36.964854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da61eb0-6028-4f39-81dc-c178ca0d5a09 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:36.994331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.009026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.009790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.010066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.010327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.010965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.016312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:37.027177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:37.029137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:37.030781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 518/996] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Aug 1 16:26:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:37.036181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.054756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df51052-73fe-4540-a90a-b8626cfab01d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.080009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.092839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.093929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.094205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.094472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.095152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:37.095569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.096875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218ac29-0ea7-40bd-b421-9078b759443b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.102004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 479/997] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:37.103145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4c8d6-be45-4133-9952-0f5fa3592622 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.103501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4c8d6-be45-4133-9952-0f5fa3592622 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.103883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4c8d6-be45-4133-9952-0f5fa3592622 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.105167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 480/998] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:37.108835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4c8d6-be45-4133-9952-0f5fa3592622 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.147619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc803e-ff67-4d91-aded-68cdbaa87fdb None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.181428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.182225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.182562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.182832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.183501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:37.183825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.185197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9025224-2b93-4df1-a124-a610a8f1e63e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.190085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 519/999] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:37.191356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.191634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.191936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.210123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:37.211355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:37.214572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4c8d6-be45-4133-9952-0f5fa3592622 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.261481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.262427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.263567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.264238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.265008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.266397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] RBAC: Adding query filter params (interface=public, service_id=533d7f5d4bbb4ca299203863653a5b0b, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:37.267048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.269645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.278679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 481/1000] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/endpoints?service_id=533d7f5d4bbb4ca299203863653a5b0b&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:37.280532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.281935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 520/1001] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:37.284025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.336741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.388914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c18c-292d-4640-b2dd-7d8c8215b26c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.397190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c28d8-aa1e-404d-ab83-4a4775ab114b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.425408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.426194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.426518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.426828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.427676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:37.428163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.429914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.435803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 482/1002] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:37.436669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.437379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.438014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.438288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.438552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.439235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.440747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.447800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 521/1003] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:26:37.448634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.507015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.569583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856bfe5-c8ec-40f8-8497-002918b4b6e0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.575665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaba81e-eeb4-43cf-8cc3-9412984564ef None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.594575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.607495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.608276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.608875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.608991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.609642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.611027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.623102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.623102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.623503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.624792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] PATH_INFO: `/v3/services/image` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.624792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.627673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.635317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 522/1004] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:37.638703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:37.640001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:37.641054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 483/1005] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/services/image => generated 86 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:37.646558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.782071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e87da2-1767-4962-bab8-5353fed4ebd0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.783944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:37.824689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.825308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.825572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.825817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.826460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:37.827689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.829197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6cd93-5dd7-40a4-8c21-1298a3c485f2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:37.837125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 523/1006] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:37.844721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:37.956989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27484ad-226f-4c5d-99c7-5896c3f6f406 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:37.993617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:37.994237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:37.994516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:37.994782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:37.995463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] RBAC: Adding query filter params (type=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:37.995774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:37.997512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f65ea1-ad0b-4817-b78e-370d79a39d5b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:38.007862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 484/1007] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:39.713639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706107f-4955-4201-a982-d58feecda9c5 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:39.714157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706107f-4955-4201-a982-d58feecda9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:39.714266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706107f-4955-4201-a982-d58feecda9c5 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:39.715582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 524/1008] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:39.718983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187a55-31d2-4724-97d1-6eaa128eb513 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:39.719553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187a55-31d2-4724-97d1-6eaa128eb513 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:39.719553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6187a55-31d2-4724-97d1-6eaa128eb513 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:39.721120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 485/1009] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:39.721478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:39.721879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:39.722375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:39.726056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:39.726336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:39.726607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:39.742774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:39.744017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:39.749309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:39.750501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:39.801913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:39.827250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 525/1010] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:39.842491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 486/1011] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:39.851938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:39.865253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:39.973472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:39.980670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa28f6-ce8d-4efb-9720-ab7279febdab None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:39.986087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633d7db-bd99-418d-8f13-5b572e82b5a7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:40.022461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d8a5f-4a82-4c97-8674-06f496fa2d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.023124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d8a5f-4a82-4c97-8674-06f496fa2d76 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.023393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d8a5f-4a82-4c97-8674-06f496fa2d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.023663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d8a5f-4a82-4c97-8674-06f496fa2d76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.024872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d8a5f-4a82-4c97-8674-06f496fa2d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.026312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d8a5f-4a82-4c97-8674-06f496fa2d76 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.027710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.028278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 526/1012] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:40.028393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.028641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.028904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.029242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29a968-83fb-4ce2-9f5b-88f3a6753759 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.029505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29a968-83fb-4ce2-9f5b-88f3a6753759 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.030000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29a968-83fb-4ce2-9f5b-88f3a6753759 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.030078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.031426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 527/1013] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:40.031532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.036335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29a968-83fb-4ce2-9f5b-88f3a6753759 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-cd5e2e8d-afef-4fc2-accc-0fec7d346712 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:40.040788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:40.042173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:40.044726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 487/1014] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:40.046931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.046931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.046931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.063243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:40.064528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:40.092880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:40.134316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 488/1015] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:40.135161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:40.153300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29a968-83fb-4ce2-9f5b-88f3a6753759 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:40.190468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.191087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.191371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.191637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.192361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.193805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.223151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 528/1016] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:40 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:40.224141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:40.244109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:40.260314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5057a3e-cc92-42c0-8a8f-83f494141628 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:40.315921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.316579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.316886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.317149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.318826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:40.319169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.320931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.328261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 489/1017] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:40.334391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:40.349335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365375d-66ba-491b-8723-d48815d17ea0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:40.369215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:40.396844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.398972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.398972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.398972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.399522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.400913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.427696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 529/1018] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:26:40 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:40.473421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab70aa2-ba87-4acb-a836-e2f35985c6a6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:40.522182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.522820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.523097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.523361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.524075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:40.524376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.525761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a14b2-9293-44ee-9905-3160f02b547e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.532245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 490/1019] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Aug 1 16:26:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:40.538617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:40.633931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:40.662997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366174a-feb8-4f02-9fc4-7f21145d04c3 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:40.707863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf2eaa-1d30-4145-96d4-ed085067aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:40.708448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2eaa-1d30-4145-96d4-ed085067aad0 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:40.708718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2eaa-1d30-4145-96d4-ed085067aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:40.708989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf2eaa-1d30-4145-96d4-ed085067aad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:40.709699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2eaa-1d30-4145-96d4-ed085067aad0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:40.711076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf2eaa-1d30-4145-96d4-ed085067aad0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:40.733039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 530/1020] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:42.389382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105129b9-8503-4209-864f-0118a89f1bef None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.389943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105129b9-8503-4209-864f-0118a89f1bef None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.390224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105129b9-8503-4209-864f-0118a89f1bef None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.393268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 491/1021] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:42.398605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.399057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.399144 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.418445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:42.419637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:42.461053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e194f26-57b6-424d-9e67-e96eead9da20 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.461401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e194f26-57b6-424d-9e67-e96eead9da20 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.461594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e194f26-57b6-424d-9e67-e96eead9da20 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.462957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 492/1022] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:42.468271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.468547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.468824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.486548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:42.487719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:42.499561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 531/1023] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:42.524512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:42.564574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 493/1024] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:42.588114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:42.590810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:42.617285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:42.635499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120402-f206-4e8c-b945-3dd1aeb18729 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:42.684277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:42.684970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.685229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.685492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.687123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:42.688693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:42.690435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 532/1025] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:42.696044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:42.718711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9ca19-a503-42c2-a40e-490d25a970d4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:42.761112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:42.761762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.762038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.762301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] PATH_INFO: `/v3/services/image` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.762921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:42.764464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:42.774405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:42.775457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:42.776703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 494/1026] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Aug 1 16:26:42 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:42.782005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:42.813191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db7f2-59e1-4915-974f-4887bc1cc3b0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:42.843528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:42.853196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ad4eb-65c5-4a33-bd79-56b3f4ce8cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:42.853832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ad4eb-65c5-4a33-bd79-56b3f4ce8cad None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.854106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ad4eb-65c5-4a33-bd79-56b3f4ce8cad None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.854378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ad4eb-65c5-4a33-bd79-56b3f4ce8cad None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.855074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ad4eb-65c5-4a33-bd79-56b3f4ce8cad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:42.856572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ad4eb-65c5-4a33-bd79-56b3f4ce8cad None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:42.872297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:42.884073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 533/1027] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:26:42 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:26:42.896409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28597210-c171-4438-92c4-efdf96f1d60b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:42.919947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f9c6-f36b-4ec8-8519-81c9b6dd9472 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.920353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f9c6-f36b-4ec8-8519-81c9b6dd9472 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.920566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f9c6-f36b-4ec8-8519-81c9b6dd9472 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.922570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 534/1028] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:42.928111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.928484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.928655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.949893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:42.951119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:42.952961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:42.953603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:42.953879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:42.954146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:42.954845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] RBAC: Adding query filter params (name=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:42.955149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:42.956609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:42.963033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 495/1029] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:42.969244 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.055794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 535/1030] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:43.075347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.114595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bfac1-bc2a-4493-92f4-b229669fa944 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.152078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.152704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.152982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.153420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.154105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] RBAC: Adding query filter params (type=image) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:43.155154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:43.156633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:43.162445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 496/1031] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:43.168773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.185300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:43.190948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e75b41-c39d-473e-bb03-647b0a6f249e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.205614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:43.231032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.231971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.232398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.232701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.243727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:43.252162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:43.253476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:43.254301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 536/1032] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:43.257058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.297190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c73a8-dc00-4777-984b-330d87946e78 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.354849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41152fda-800c-4900-a773-535184aadc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.355603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41152fda-800c-4900-a773-535184aadc51 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.355909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41152fda-800c-4900-a773-535184aadc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.356182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41152fda-800c-4900-a773-535184aadc51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.356943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41152fda-800c-4900-a773-535184aadc51 None admin] RBAC: Adding query filter params (interface=public, service_id=3b72aefaa37646a483abc279f7f38e14, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:43.357251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41152fda-800c-4900-a773-535184aadc51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:43.359020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41152fda-800c-4900-a773-535184aadc51 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:43.370034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 497/1033] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/endpoints?service_id=3b72aefaa37646a483abc279f7f38e14&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:43.370807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41152fda-800c-4900-a773-535184aadc51 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.384650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae0181-fb85-4170-9387-a298d0f2f5e2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.421227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.422557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.422833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.423104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.423861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:43.424164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:43.425584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:43.439104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 537/1034] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:43.440190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.461336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:43.477080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:43.486343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41152fda-800c-4900-a773-535184aadc51 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.531839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.532781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.533270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.533773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.535155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:43.536568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:43.547206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 498/1035] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:26:43.549529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c29ce-e799-463c-b2b9-859b8355bd65 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.589197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.589843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.590088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.590376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.606425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:43.608325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:43.625428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:43.626643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:43.628307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:43.628307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:43.628307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:43.628307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:43.628307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 538/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:43.632997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.762888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c805a-c7e6-42f6-91e9-f0663379d810 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.799008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.799646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.799951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.800224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:43.800924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:43.801226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:43.802616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:43.812486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:43.818393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 499/1037] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:43.824647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:43.928727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:43.946072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e77c8-f01a-417a-b3f2-5688b34b8c62 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:43.988219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:43.988976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:43.989346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:43.989653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:44.000580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:44.002279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:44.017193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:44.018241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:44.019212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:44.021220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 539/1038] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:26:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:44.026541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:44.135573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0ef1d-10d1-48f4-9616-839900c8e962 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:44.171277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:44.171983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:44.172260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:44.172530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:44.173326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:44.173627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:44.175076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:44.193970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 500/1039] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:26:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:44.199682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:44.311734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cbe7-fc6a-4fda-a981-ac0207515015 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:44.347127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:44.347745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:44.348043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:44.348312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:44.363694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:44.417522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:44.418989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:44.529202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 540/1040] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:44.531220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:44.592312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:44.637638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fe4a9-d36c-4ab9-8aa2-e1abd0e1daea None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:44.673652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:44.676887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:44.676887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:44.676887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:44.690582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:44.692134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:44.709771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:44.710778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:44.711702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:44.711702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:44.711702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:44.711702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:44.711702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 501/1041] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:44.717641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:44.776318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:44.842714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277a55-78c4-4833-8145-2d72706170c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:44.887478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:44.888129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:44.888404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:44.888779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:44.889680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:44.891846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:44.891846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348836-49c9-4bda-ba77-0e97db5a244f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:44.904231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 541/1042] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:44.910380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.007518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f31d8-841e-499b-b374-3bae02fec0fb None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.007919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f31d8-841e-499b-b374-3bae02fec0fb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.008112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f31d8-841e-499b-b374-3bae02fec0fb None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.009699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 542/1043] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:45.014845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.014999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.015221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.029176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c5f5a-b434-4a7a-a8c4-d1e3db5a6d70 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.034932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:45.036251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:45.041633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.071389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.072087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.072336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.072634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.081653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.083174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:45.096512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:45.097477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:45.098404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:45.102665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 502/1044] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:45.103602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.117046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 543/1045] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:45.140988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.187132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.242897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc4844-a3b0-4c12-9489-658877e95ed0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.267223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bcdbe-0895-44f8-9170-55d203dd0af2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.283132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.283811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.284106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.284359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.285118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:45.285423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.286889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:45.301021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 503/1046] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:45.309095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.319690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.320447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.320680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.320944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.335743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:45.340724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:45.341668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:45.342603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 544/1047] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:45.349938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.362653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.451838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0857847-5f5d-4587-acca-93a800afd1ac None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.471619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.483521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c47f-b2a4-49b4-8a69-5fe6ab1b3781 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.497335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.498206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.498451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.498786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.499636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] RBAC: Adding query filter params (role.id=0fdcdf07d33844aebfc4c6168ec857ed, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=dd1da21c00d8494587e3ede50a5a365b) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:45.500039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.501534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:45.507366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 504/1048] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/role_assignments?user.id=dd1da21c00d8494587e3ede50a5a365b&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=0fdcdf07d33844aebfc4c6168ec857ed => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:45.523400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.524248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.524618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.524982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.525829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:45.526230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.527756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15621680-5c5e-4065-95e6-877db4d19a61 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:45.542679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 545/1049] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:45.543991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.649923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.658657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866ba8a-52eb-4220-9c04-afcea13f22c6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.685641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ed07d-7680-406d-8052-d637208ebfb2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.685919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ed07d-7680-406d-8052-d637208ebfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.686194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ed07d-7680-406d-8052-d637208ebfb2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.692455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 546/1050] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:26:45.692865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.693122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.693388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.714199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:45.715256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.715942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.716221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.716418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:45.716535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] PATH_INFO: `/v3/users/11a6dba7e4574bddb1083e70140973c1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.732084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] RBAC: Authorizing `identity:get_user(user_id=11a6dba7e4574bddb1083e70140973c1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.733928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:45.735686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 505/1051] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/users/11a6dba7e4574bddb1083e70140973c1 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:45.741981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.807625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 547/1052] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:45.830708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:45.860624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e0f1-54d4-435d-9b64-a1c6fe17b1c0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.898266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.898890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.899165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.899432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.928638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.933605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:45.948976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e13fd-c349-4839-8a95-74eb7bd3bf23 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:45.978034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.979650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:45.992464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:45.993136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:45.993412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:45.993767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:45.994406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:45.996056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:46.010035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:46.011145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:46.012031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 548/1053] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:46.018157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:46.112349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 506/1054] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:26:46.118228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:46.150263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e802372-11ec-41ff-b1df-91a156af20d4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:46.176831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:46.189060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:46.189610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:46.189899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:46.190260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:46.190837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:46.191136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:46.192684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:46.202279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 549/1055] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:46.203951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:46.223471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:46.231024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04d317-54df-4c32-9f92-370266f27397 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:46.271100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:46.271755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:46.272071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:46.272343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:46.273585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] RBAC: Authorizing `identity:get_project(project_id=819bdc4863854783bced3bcb1aeea3d1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:46.275034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:46.282904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 507/1056] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:26:46 2022] GET /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:46.284061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:46.317303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c1af0-d7f1-4341-9c17-2b4ed4fb32d8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:46.352397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:46.353019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:46.353296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:46.353566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:46.354245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] RBAC: Adding query filter params (type=image) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:46.354549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:46.355953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:46.367358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 550/1057] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:46.368614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:46.413341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67a8c3-eca7-4859-9847-6395f74d45bf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:46.445345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:46.470896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:46.471547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:46.471847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:46.472126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:46.472932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=819bdc4863854783bced3bcb1aeea3d1, user.id=11a6dba7e4574bddb1083e70140973c1) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:46.473244 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:46.474719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ce94b-0b32-4f4e-a5c4-7b8d61329702 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:46.481106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 508/1058] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:46 2022] GET /identity/v3/role_assignments?user.id=11a6dba7e4574bddb1083e70140973c1&scope.project.id=819bdc4863854783bced3bcb1aeea3d1&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:46.496758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:46.523813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0dcb8-7830-4177-82c4-121fb13d4c81 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:46.564706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab060f-d80c-4906-93a2-26a81bcab924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:46.565341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab060f-d80c-4906-93a2-26a81bcab924 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:46.565608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab060f-d80c-4906-93a2-26a81bcab924 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:46.565889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab060f-d80c-4906-93a2-26a81bcab924 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:46.566708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab060f-d80c-4906-93a2-26a81bcab924 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:46.568167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab060f-d80c-4906-93a2-26a81bcab924 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:46.593332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 551/1059] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:26:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:26:47.507111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627b69a-0f46-4ee9-bf09-cf6417e6f3a6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:47.507665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627b69a-0f46-4ee9-bf09-cf6417e6f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:47.507665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627b69a-0f46-4ee9-bf09-cf6417e6f3a6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:47.509005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 509/1060] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:47.514477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:47.514837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:47.515204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:47.533979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:47.535152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:47.620146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 552/1061] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:47.639862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627b69a-0f46-4ee9-bf09-cf6417e6f3a6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:47.776544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627b69a-0f46-4ee9-bf09-cf6417e6f3a6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:47.817709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:47.818753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:47.819224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:47.819679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:47.833852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:47.838817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:47.840037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:47.846298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 510/1062] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 16:26:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:47.848656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:47.913363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:47.958981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fceb0cf-864e-44ff-871e-e9f8a97718bf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.000190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.001151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.001426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.001697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.002420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:48.002719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:48.004112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:48.014133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 553/1063] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:26:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:48.020051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.071653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:48.149162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880f83-7805-4e20-87ef-d775984afc0b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.187983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.188634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.188912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.189179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.200650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:48.202580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:48.218614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:48.219641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:48.220919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:48.220919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:48.220919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:48.220919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:48.224703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 511/1064] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:48.226232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.334367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77badd0-11f5-41eb-bdbd-3308db233d96 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.358534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:48.370573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.371536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.371847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.372115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.372818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:48.373163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:48.375017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be786-f0b2-4c9f-af59-56e85e23f53f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:48.387490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 554/1065] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:48.394009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.494759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:48.502854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a1d66-3efd-45df-b370-f8a06d3e8b58 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.523691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c78e85-4f9b-4d3f-bb56-5095ec188162 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.523996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c78e85-4f9b-4d3f-bb56-5095ec188162 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.524276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c78e85-4f9b-4d3f-bb56-5095ec188162 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.525700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 555/1066] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:48.530802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.531072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.531358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.540010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.540662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.540940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.541210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.549849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:48.550073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:48.551050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:48.551486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:48.564507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:48.565536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:48.566507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 512/1067] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:48.571736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.631916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 556/1068] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:48.650970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.691426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c7967-0509-4f4e-bd4d-81f66a6416b4 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.727258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.727914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.728189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.728452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.729239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:48.729536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:48.730966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948912fa-5852-4f2e-a29f-01125ae6b2ee None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:48.748756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 513/1069] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:48.749775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1659-a242-46c6-8600-fde6e3a770d0 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.750047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1659-a242-46c6-8600-fde6e3a770d0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.750314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1659-a242-46c6-8600-fde6e3a770d0 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.751964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 514/1070] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:48.754581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1659-a242-46c6-8600-fde6e3a770d0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.757768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362cd8-cec7-43e8-84a7-6fef706227d1 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.772271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:48.773337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:48.799695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb0065-fdd3-4e04-854f-304c90b00d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.800385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb0065-fdd3-4e04-854f-304c90b00d39 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.800656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb0065-fdd3-4e04-854f-304c90b00d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.800926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb0065-fdd3-4e04-854f-304c90b00d39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.810946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb0065-fdd3-4e04-854f-304c90b00d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-5bfb0065-fdd3-4e04-854f-304c90b00d39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:48.816138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:48.817156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:48.818006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 557/1071] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:48.818707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.818707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.819028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.837217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:48.838461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:48.861740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1659-a242-46c6-8600-fde6e3a770d0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:48.894800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:48.895341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:48.895622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:48.895914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:48.906109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:48.907588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 558/1072] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:48.907783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:48.908259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:48.923039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:48.924140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:48.929447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:48.929447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:48.929447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:48.929447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:48.929447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 515/1073] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:48.929447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:48.995740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.017706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e39bbb-9d8c-4b20-a25d-68752107d6a2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.020405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.031534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19239c-2552-4081-8deb-7ac5723a9079 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.053078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.053716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.054026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.054299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.055012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:49.055307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.056703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.065074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 559/1074] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:49.065853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.065957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.066517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.066794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.067063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.133966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.135423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.175169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbcaec-3599-40aa-96c4-0242c9753960 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.199388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.209298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.209926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.210199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.210469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.211167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:49.211469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.212887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.225428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 560/1075] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:49.226116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.238760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 516/1076] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:26:49.238760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.257684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.331868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a1907-1e47-42bd-ad97-ab635daf411e None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.336335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437943b5-b5d3-431d-b4f1-9fcfc4883a9f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.366258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.366901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.367177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.367444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] PATH_INFO: `/v3/users/11a6dba7e4574bddb1083e70140973c1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.368630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] RBAC: Authorizing `identity:get_user(user_id=11a6dba7e4574bddb1083e70140973c1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.370081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.370383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.371349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.371640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.371940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.372081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 561/1077] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/users/11a6dba7e4574bddb1083e70140973c1 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:49.372987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.379168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.380645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:49.392036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:49.392944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:49.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 517/1078] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:49.394351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.422084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.473174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.480900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84271f8f-8e97-4a74-8655-7a37d14dbed4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.495671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05288bf-90cb-40f2-9f2e-db705fa7f6db None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.518117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.518781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.519051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.519315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.529137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.531703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.532609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82855892-496a-43d7-86de-658655b8829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.533261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82855892-496a-43d7-86de-658655b8829c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.533540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82855892-496a-43d7-86de-658655b8829c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.533810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82855892-496a-43d7-86de-658655b8829c None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.535020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82855892-496a-43d7-86de-658655b8829c None admin] RBAC: Authorizing `identity:get_project(project_id=819bdc4863854783bced3bcb1aeea3d1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.536485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82855892-496a-43d7-86de-658655b8829c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.538222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 518/1079] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:49.538976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82855892-496a-43d7-86de-658655b8829c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:49.550635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:49.551691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:49.552736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:49.552736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:49.552736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:49.552736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:49.552736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 562/1080] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:49.552736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.641333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.649722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82855892-496a-43d7-86de-658655b8829c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.652599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903cf21-c8cc-4838-9b56-cf203a6f71fc None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.687017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.687685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.687982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.688251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1/users/11a6dba7e4574bddb1083e70140973c1/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.690168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] RBAC: Authorizing `identity:create_grant(project_id=819bdc4863854783bced3bcb1aeea3d1, user_id=11a6dba7e4574bddb1083e70140973c1, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.691128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.691602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.691805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.692089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.692362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.693113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:49.693422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.694846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.705497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 563/1081] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:26:49 2022] PUT /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1/users/11a6dba7e4574bddb1083e70140973c1/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:26:49.706540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.708617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 519/1082] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:49.715058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:49.716932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:49.824560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d7b73-18b6-4b6a-875f-2cab8cf089ec None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.833297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee0eee-bf18-4868-8d08-b909811e1029 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:49.862624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.863263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.863534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.863823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.864522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:49.864835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.866197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b27f-7d6a-4738-9f00-25f9826de149 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.877780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e609dc7-f600-4b81-8c81-4cc6f3fb850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:49.878493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609dc7-f600-4b81-8c81-4cc6f3fb850f None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:49.878805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609dc7-f600-4b81-8c81-4cc6f3fb850f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:49.879078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e609dc7-f600-4b81-8c81-4cc6f3fb850f None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/dd1da21c00d8494587e3ede50a5a365b/roles/0fdcdf07d33844aebfc4c6168ec857ed` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:49.881519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 564/1083] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:49.882102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609dc7-f600-4b81-8c81-4cc6f3fb850f None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=dd1da21c00d8494587e3ede50a5a365b, role_id=0fdcdf07d33844aebfc4c6168ec857ed)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:49.883583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e609dc7-f600-4b81-8c81-4cc6f3fb850f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:49.900701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 520/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:26:49 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/dd1da21c00d8494587e3ede50a5a365b/roles/0fdcdf07d33844aebfc4c6168ec857ed => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:26:51.745715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc73b2-4b9d-41a0-995c-e0ca6d2fd90c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:51.746663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc73b2-4b9d-41a0-995c-e0ca6d2fd90c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:51.746663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc73b2-4b9d-41a0-995c-e0ca6d2fd90c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:51.748482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 565/1085] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:51.753983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff8d9-924f-4d81-98cb-676f817ca197 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:51.754420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff8d9-924f-4d81-98cb-676f817ca197 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:51.754521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff8d9-924f-4d81-98cb-676f817ca197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:51.773226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a7fff8d9-924f-4d81-98cb-676f817ca197 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:51.774421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a7fff8d9-924f-4d81-98cb-676f817ca197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:51.784406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:51.849891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 521/1086] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:51.873231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc73b2-4b9d-41a0-995c-e0ca6d2fd90c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:51.943566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:51.966705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac227-9932-4bda-9857-6ff83328ed6d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:51.967096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac227-9932-4bda-9857-6ff83328ed6d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:51.967449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac227-9932-4bda-9857-6ff83328ed6d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:51.969104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 522/1087] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:51.974388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:51.974647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:51.974917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:51.981552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc73b2-4b9d-41a0-995c-e0ca6d2fd90c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:51.994944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:51.996515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:52.025068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.025428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.025716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.025990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.039620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:52.044520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:52.046105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:26:52.047183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 566/1088] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:52.054033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.072545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 523/1089] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:26:52.105371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.174763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5200e01-4a50-4117-a145-49a184d8038d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.189328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.204431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.213809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.214439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.214715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.214988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.215829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:52.216133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.216842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce261fa-37e8-421a-8b14-e89e318b88aa None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.217776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.226411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 567/1090] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:52.232389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.256694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.257428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.257709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.257987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.269065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:26:52.274058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:52.275211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:52.276121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 524/1091] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:52.281759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.346931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea9e85-d61e-4e07-b3e2-1220679fac90 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.385711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.386374 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.386675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.386928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] PATH_INFO: `/v3/users/11a6dba7e4574bddb1083e70140973c1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.388215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] RBAC: Authorizing `identity:get_user(user_id=11a6dba7e4574bddb1083e70140973c1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.390036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.391744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 568/1092] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/users/11a6dba7e4574bddb1083e70140973c1 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:52.397280 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403069-6f37-4994-9754-0f1e082db5fa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.397604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.436228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.436857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.437131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.437397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.438115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:52.438421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.439834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.441511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.450182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 525/1093] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:52.458804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.470086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.520806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d61a45-4ae3-462c-986e-ebe1e58879e9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.559087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.559711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.560026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.560308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.586630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ae8a6-e93e-4df1-ab5d-00082047b203 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.624266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.624928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.625201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.625462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.638417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.642008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.660308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:52.661715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:52.662671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:52.663574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:52.663574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:52.663574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:52.663574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:52.663892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.663973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 526/1094] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:52.668862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.721307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.722224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.767449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 569/1095] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:26:52.773487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.775356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edf3e5-a915-4baa-9db6-974415192926 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.811879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.812482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.812757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.813025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.813721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:52.814024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.815413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.828391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 527/1096] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:52.834744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.875151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ab300-ce69-485b-a8e7-fab3801f7e07 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.917742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.918392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.918671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.918938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.920133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] RBAC: Authorizing `identity:get_project(project_id=819bdc4863854783bced3bcb1aeea3d1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.921589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:52.923283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 570/1097] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:52.929438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:52.944674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0d98a-7405-4f34-a404-62deee81f2df None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:52.954037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.961923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:52.983438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:52.983438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:52.983438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:52.984076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:52.992201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:52.993681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:26:53.007040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:53.008771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:53.009777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 528/1098] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:26:53.014436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:53.049825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4a2fd-1e70-4115-9465-3d6c5fc8b9aa None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:53.083641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:53.084276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:53.084545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:53.084808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:53.085581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=819bdc4863854783bced3bcb1aeea3d1, user.id=11a6dba7e4574bddb1083e70140973c1) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:53.085881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:53.087300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:53.093471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 571/1099] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:52 2022] GET /identity/v3/role_assignments?user.id=11a6dba7e4574bddb1083e70140973c1&scope.project.id=819bdc4863854783bced3bcb1aeea3d1&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:53.123954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d50b39-b6b7-46b1-9621-930008aa49aa None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:53.164583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21071068-622a-4eed-802e-5665fb31928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:53.165231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071068-622a-4eed-802e-5665fb31928e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:53.165509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071068-622a-4eed-802e-5665fb31928e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:53.165781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071068-622a-4eed-802e-5665fb31928e None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:53.166592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21071068-622a-4eed-802e-5665fb31928e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:53.166900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21071068-622a-4eed-802e-5665fb31928e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:53.168414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21071068-622a-4eed-802e-5665fb31928e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:53.191329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 529/1100] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Aug 1 16:26:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:26:53.197660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:53.305411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:53.318665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39b924-7d46-4308-9125-eb4811e15ab2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:53.354292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:53.354908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:53.355161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:53.355428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:53.430569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:53.432052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:53.521719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:53.538770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 572/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:53.544669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21071068-622a-4eed-802e-5665fb31928e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:53.578806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:53.660867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21071068-622a-4eed-802e-5665fb31928e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:53.700871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:53.701520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:53.701790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:53.702056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:53.712974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:53.714416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:26:53.729586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:53.730512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:53.731394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:26:53.731394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:26:53.731394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:26:53.731394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:53.731394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 530/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:53.736802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:53.844448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501a5f6-c510-47d7-8194-fab1cf93d5d4 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:53.881166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:53.881166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:53.881166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:53.881166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:53.885382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:53.885382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:53.885382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:53.899243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 573/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:53.905024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:53.987675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:54.010038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12059b-a95c-49ef-b8ac-0d9a47470cdf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:54.043590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:54.044262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:54.044758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:54.045037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:54.051807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:54.053286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:26:54.064576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:26:54.065506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:26:54.066402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 531/1104] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:26:54.071857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:54.122475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:54.177362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f4e6b-1119-4efd-9aeb-652ea7272ecd None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:54.219304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:54.219958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:54.220231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:54.220501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:54.221245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:54.221549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:54.222984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd0cf7-dd51-48b4-9143-2a599701c9a9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:54.236123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 574/1105] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:54.242420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:54.353366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c737f-e45f-4f01-acd7-a6052fa94b23 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:54.399564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:54.400289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:54.400609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:54.400922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:54.401763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] RBAC: Adding query filter params (role.id=0fdcdf07d33844aebfc4c6168ec857ed, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=dd1da21c00d8494587e3ede50a5a365b) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:54.402114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:54.403596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c479939-578b-446b-94b4-ad707d6765fa None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:54.412242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 532/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:54 2022] GET /identity/v3/role_assignments?user.id=dd1da21c00d8494587e3ede50a5a365b&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=0fdcdf07d33844aebfc4c6168ec857ed => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:55.229945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97781b-9a3a-454a-abd2-3d5606797245 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:55.230513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97781b-9a3a-454a-abd2-3d5606797245 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:55.230513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97781b-9a3a-454a-abd2-3d5606797245 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:55.232338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 575/1107] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:55.239194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:55.239646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:55.239749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:55.248037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:55.262929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:55.264502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:55.355617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 533/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:55.385997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97781b-9a3a-454a-abd2-3d5606797245 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:55.503847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97781b-9a3a-454a-abd2-3d5606797245 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:55.514903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:55.548099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e506a-f361-4a5b-bc5a-a0442510bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:55.548767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e506a-f361-4a5b-bc5a-a0442510bd12 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:55.549039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e506a-f361-4a5b-bc5a-a0442510bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:55.549306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e506a-f361-4a5b-bc5a-a0442510bd12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:55.550565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e506a-f361-4a5b-bc5a-a0442510bd12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:55.552313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e506a-f361-4a5b-bc5a-a0442510bd12 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:55.554248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 576/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:26:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:55.562086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:55.684747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9450a1-9eba-440e-9ad2-3f1e8e6a8886 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:55.718127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:55.723749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c018031-97b5-4ce6-bf98-1c7a8f638c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:55.724398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c018031-97b5-4ce6-bf98-1c7a8f638c87 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:55.724670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c018031-97b5-4ce6-bf98-1c7a8f638c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:55.724936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c018031-97b5-4ce6-bf98-1c7a8f638c87 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:55.725632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c018031-97b5-4ce6-bf98-1c7a8f638c87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:55.727068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c018031-97b5-4ce6-bf98-1c7a8f638c87 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:55.749215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 534/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:26:55 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:26:57.794280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ed80-adde-46de-8937-bdaed9f05be3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:57.794969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ed80-adde-46de-8937-bdaed9f05be3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:57.794969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ed80-adde-46de-8937-bdaed9f05be3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:57.796371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 577/1111] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:26:57.801655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:57.802085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:57.802191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:57.821482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:26:57.822663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:26:57.897712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 535/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:26:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:26:57.923352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256ed80-adde-46de-8937-bdaed9f05be3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:58.035420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256ed80-adde-46de-8937-bdaed9f05be3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:58.071592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:58.072265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:58.072540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:58.072813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] PATH_INFO: `/v3/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:58.074338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] RBAC: Authorizing `identity:get_role(role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:58.076140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:58.077529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 578/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:26:57 2022] GET /identity/v3/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:58.083794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:58.196319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5effc-37a7-469a-b193-0595a64c08c3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:58.214377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:58.232378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:58.232981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:58.233254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:58.233518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] PATH_INFO: `/v3/users/0f3f7024fb054c92a37958764a56a082` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:58.247277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] RBAC: Authorizing `identity:get_user(user_id=0f3f7024fb054c92a37958764a56a082)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:58.248951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:58.250646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 536/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:26:58 2022] GET /identity/v3/users/0f3f7024fb054c92a37958764a56a082 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:58.257057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:58.305393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:58.364020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96d2af-c3b5-488b-9ec8-97a805c916c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:58.397862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:58.398476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:58.398754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:58.399025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:58.475096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:58.479806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:58.526217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:26:58.584355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 579/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:26:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:26:58.590289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:58.691156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff7d2d-47e0-4698-bfc4-6c2872919e41 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:58.726528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24f559-65c5-4baa-90fb-ecb13dcdb7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:58.727158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24f559-65c5-4baa-90fb-ecb13dcdb7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:58.727432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24f559-65c5-4baa-90fb-ecb13dcdb7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:58.727700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24f559-65c5-4baa-90fb-ecb13dcdb7b3 None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:58.728908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24f559-65c5-4baa-90fb-ecb13dcdb7b3 None admin] RBAC: Authorizing `identity:get_project(project_id=819bdc4863854783bced3bcb1aeea3d1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:58.730344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24f559-65c5-4baa-90fb-ecb13dcdb7b3 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:58.732136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 537/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:26:58 2022] GET /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:26:58.738497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:26:58.852286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec04c0-d495-40a3-abbb-8919797ce4c8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:26:58.888404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:26:58.889092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:26:58.889416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:26:58.889734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:26:58.890641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] RBAC: Adding query filter params (role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=819bdc4863854783bced3bcb1aeea3d1, user.id=0f3f7024fb054c92a37958764a56a082) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:26:58.891056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:26:58.892662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7af1-0b9d-479b-b39f-349bf7486eb6 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:26:58.899204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 580/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:26:58 2022] GET /identity/v3/role_assignments?user.id=0f3f7024fb054c92a37958764a56a082&scope.project.id=819bdc4863854783bced3bcb1aeea3d1&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:00.873044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29591-cf63-4a56-ab92-1da59d20c19b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:00.873600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29591-cf63-4a56-ab92-1da59d20c19b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:00.873600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29591-cf63-4a56-ab92-1da59d20c19b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:00.874968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 538/1118] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:00.880457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e95308-2831-4f15-97ed-84129192179f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:00.880935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e95308-2831-4f15-97ed-84129192179f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:00.881032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e95308-2831-4f15-97ed-84129192179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:00.900672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d8e95308-2831-4f15-97ed-84129192179f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:00.901880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d8e95308-2831-4f15-97ed-84129192179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:00.904017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:00.976068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 581/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:00.999108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29591-cf63-4a56-ab92-1da59d20c19b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:01.047575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:01.105660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29591-cf63-4a56-ab92-1da59d20c19b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:01.142937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:01.143589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:01.143980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:01.144157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] PATH_INFO: `/v3/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:01.145627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] RBAC: Authorizing `identity:get_role(role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:01.147048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:01.148445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 539/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:00 2022] GET /identity/v3/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:01.156452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e95308-2831-4f15-97ed-84129192179f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:01.270463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e95308-2831-4f15-97ed-84129192179f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:01.309343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:01.310026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:01.310293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:01.310655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] PATH_INFO: `/v3/users/0f3f7024fb054c92a37958764a56a082` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:01.311915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] RBAC: Authorizing `identity:get_user(user_id=0f3f7024fb054c92a37958764a56a082)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:01.313485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:01.315271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 582/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:01 2022] GET /identity/v3/users/0f3f7024fb054c92a37958764a56a082 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:01.323425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:01.442275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454900c-4b54-46fe-a68e-3e96275ecb68 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:01.456369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:01.480033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c96fb-54cc-4a16-9c19-ec27cc299094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:01.480720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c96fb-54cc-4a16-9c19-ec27cc299094 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:01.481088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c96fb-54cc-4a16-9c19-ec27cc299094 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:01.481361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c96fb-54cc-4a16-9c19-ec27cc299094 None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:01.483084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c96fb-54cc-4a16-9c19-ec27cc299094 None admin] RBAC: Authorizing `identity:get_project(project_id=819bdc4863854783bced3bcb1aeea3d1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:01.484839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c96fb-54cc-4a16-9c19-ec27cc299094 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:01.486918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 540/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:01 2022] GET /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:01.496321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:01.510256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:01.615402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d80a9f-5c67-4eec-8bf8-1b8426e7c3e9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:01.651465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4101-85fc-4c7d-b892-be2d77b68685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:01.652163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4101-85fc-4c7d-b892-be2d77b68685 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:01.652443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4101-85fc-4c7d-b892-be2d77b68685 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:01.652716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4101-85fc-4c7d-b892-be2d77b68685 None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1/users/0f3f7024fb054c92a37958764a56a082/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:01.655557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4101-85fc-4c7d-b892-be2d77b68685 None admin] RBAC: Authorizing `identity:create_grant(project_id=819bdc4863854783bced3bcb1aeea3d1, user_id=0f3f7024fb054c92a37958764a56a082, role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:01.657027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4101-85fc-4c7d-b892-be2d77b68685 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:01.672688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 583/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:27:01 2022] PUT /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1/users/0f3f7024fb054c92a37958764a56a082/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:27:03.729887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e80e-4374-47cf-a2a3-df3e82022c56 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:03.730429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e80e-4374-47cf-a2a3-df3e82022c56 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:03.730429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e80e-4374-47cf-a2a3-df3e82022c56 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:03.731902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 541/1124] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:03.737807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb18478-111a-4770-b14e-247659968bed None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:03.738257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb18478-111a-4770-b14e-247659968bed None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:03.738368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb18478-111a-4770-b14e-247659968bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:03.758021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-afb18478-111a-4770-b14e-247659968bed None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:03.759224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-afb18478-111a-4770-b14e-247659968bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:03.831717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:03.840449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 584/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:03.864536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e80e-4374-47cf-a2a3-df3e82022c56 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:03.977885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e80e-4374-47cf-a2a3-df3e82022c56 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:04.020171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:04.020826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:04.021105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:04.021373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] PATH_INFO: `/v3/roles/39ba989c9d764fb383e62e5fa96b1ee8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:04.022833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] RBAC: Authorizing `identity:get_role(role_id=39ba989c9d764fb383e62e5fa96b1ee8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:04.024256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:04.025664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 542/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:03 2022] GET /identity/v3/roles/39ba989c9d764fb383e62e5fa96b1ee8 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:04.040558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb18478-111a-4770-b14e-247659968bed None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:04.156122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb18478-111a-4770-b14e-247659968bed None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:04.192462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45854029-cc35-434e-8561-304f7b219773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:04.193103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854029-cc35-434e-8561-304f7b219773 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:04.193378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854029-cc35-434e-8561-304f7b219773 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:04.193649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854029-cc35-434e-8561-304f7b219773 None admin] PATH_INFO: `/v3/users/0f3f7024fb054c92a37958764a56a082` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:04.194960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45854029-cc35-434e-8561-304f7b219773 None admin] RBAC: Authorizing `identity:get_user(user_id=0f3f7024fb054c92a37958764a56a082)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:04.196436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45854029-cc35-434e-8561-304f7b219773 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:04.198104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 585/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:04 2022] GET /identity/v3/users/0f3f7024fb054c92a37958764a56a082 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:04.204984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:04.265475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:04.327596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f959-2a97-41bf-8a1d-3f995e1ed565 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:04.364429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:04.365064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:04.365350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:04.365628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:04.444933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:04.446396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:04.495937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:04.551875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 543/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:27:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:27:04.557989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45854029-cc35-434e-8561-304f7b219773 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:04.624674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:04.674469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45854029-cc35-434e-8561-304f7b219773 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:04.711316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12e279-79ed-4ee7-a755-f5226046cd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:04.711985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e279-79ed-4ee7-a755-f5226046cd3c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:04.712260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e279-79ed-4ee7-a755-f5226046cd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:04.712533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e279-79ed-4ee7-a755-f5226046cd3c None admin] PATH_INFO: `/v3/projects/819bdc4863854783bced3bcb1aeea3d1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:04.713702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e279-79ed-4ee7-a755-f5226046cd3c None admin] RBAC: Authorizing `identity:get_project(project_id=819bdc4863854783bced3bcb1aeea3d1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:04.715146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e279-79ed-4ee7-a755-f5226046cd3c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:04.717323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 586/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:04 2022] GET /identity/v3/projects/819bdc4863854783bced3bcb1aeea3d1 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:04.726370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:04.833894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144760d-5541-4310-9a44-14ec221f05a8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:04.869555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:04.870178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:04.870781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:04.871056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:04.871867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] RBAC: Adding query filter params (role.id=39ba989c9d764fb383e62e5fa96b1ee8, scope.project.id=819bdc4863854783bced3bcb1aeea3d1, user.id=0f3f7024fb054c92a37958764a56a082) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:04.872232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:04.874061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c655-1aba-4016-9d6b-e6887b3abc86 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:04.880150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 544/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:27:04 2022] GET /identity/v3/role_assignments?user.id=0f3f7024fb054c92a37958764a56a082&scope.project.id=819bdc4863854783bced3bcb1aeea3d1&role.id=39ba989c9d764fb383e62e5fa96b1ee8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:27:06.916431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df3a7-51ff-455f-a2d7-2c19b663b6dc None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:06.917003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df3a7-51ff-455f-a2d7-2c19b663b6dc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:06.917003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df3a7-51ff-455f-a2d7-2c19b663b6dc None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:06.918339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 587/1131] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:06.924235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:06.924840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:06.924933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:06.946334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:06.947455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:06.956509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:07.025072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 545/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:07.054706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df3a7-51ff-455f-a2d7-2c19b663b6dc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:07.168426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df3a7-51ff-455f-a2d7-2c19b663b6dc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:07.199744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:07.206915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141dbb-d286-483f-99c6-4567fc138c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:07.207573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141dbb-d286-483f-99c6-4567fc138c79 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:07.207884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141dbb-d286-483f-99c6-4567fc138c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:07.208155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141dbb-d286-483f-99c6-4567fc138c79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:07.209406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141dbb-d286-483f-99c6-4567fc138c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:07.210855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141dbb-d286-483f-99c6-4567fc138c79 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:07.212520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 588/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:27:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:07.219443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:07.333372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e190ac-8d8a-4602-8810-d3d31bd56684 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:07.371130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f242f77-55d3-4cde-9d04-7be7e3ecf0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:07.371808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f242f77-55d3-4cde-9d04-7be7e3ecf0da None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:07.372093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f242f77-55d3-4cde-9d04-7be7e3ecf0da None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:07.372335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f242f77-55d3-4cde-9d04-7be7e3ecf0da None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:07.373121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f242f77-55d3-4cde-9d04-7be7e3ecf0da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:07.374592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f242f77-55d3-4cde-9d04-7be7e3ecf0da None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:07.390549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:07.405998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 546/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:27:07 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:27:09.484625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67eb24-ca0f-40ad-a07e-cb4a46e68205 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:09.485394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67eb24-ca0f-40ad-a07e-cb4a46e68205 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:09.485394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67eb24-ca0f-40ad-a07e-cb4a46e68205 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:09.486519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 589/1135] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:09.492196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:09.492616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:09.492729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:09.515651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:09.516920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:09.592025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 547/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:09.618688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67eb24-ca0f-40ad-a07e-cb4a46e68205 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:09.730180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67eb24-ca0f-40ad-a07e-cb4a46e68205 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:09.770182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521420bd-e018-4b0b-be79-52160bd07478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:09.770984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521420bd-e018-4b0b-be79-52160bd07478 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:09.771102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521420bd-e018-4b0b-be79-52160bd07478 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:09.771375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521420bd-e018-4b0b-be79-52160bd07478 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:09.772625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521420bd-e018-4b0b-be79-52160bd07478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:09.774066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521420bd-e018-4b0b-be79-52160bd07478 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:09.775721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 590/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:27:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:09.782399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:09.893348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01830897-4f6e-4bf9-bc58-69db0a7d17a2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:09.919239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:09.932800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409acdca-ea10-4574-83ca-affd187f611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:09.933458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409acdca-ea10-4574-83ca-affd187f611c None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:09.933738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409acdca-ea10-4574-83ca-affd187f611c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:09.934011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409acdca-ea10-4574-83ca-affd187f611c None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:09.934881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409acdca-ea10-4574-83ca-affd187f611c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:09.936605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409acdca-ea10-4574-83ca-affd187f611c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:09.962771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 548/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:27:09 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:27:12.006827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5cc1b-2d37-4f2f-92df-9b56c7352716 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:12.007435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5cc1b-2d37-4f2f-92df-9b56c7352716 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:12.007435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5cc1b-2d37-4f2f-92df-9b56c7352716 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:12.008799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 591/1139] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:12.014922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:12.015362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:12.015479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:12.034111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:12.035277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:12.106535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 549/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:27:12.130859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5cc1b-2d37-4f2f-92df-9b56c7352716 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:12.204888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:12.246779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5cc1b-2d37-4f2f-92df-9b56c7352716 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:12.288067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:12.288711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:12.288986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:12.289257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:12.300381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:27:12.305235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:12.307028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:27:12.308568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 592/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:12.312846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:12.420572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2aa0f-46b2-4b39-9c44-c583c4ab6074 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:12.438051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:12.465667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:12.466287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:12.466565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:12.466837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:12.467563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:12.467902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:12.469290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:12.477296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 550/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:12.485125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:12.595125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb40bc7-45ac-49e6-97e4-a533afb3df60 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:12.620392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:12.630138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:12.630772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:12.631045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:12.631312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] PATH_INFO: `/v3/users/271e79eb90be41068da0e18827454fe9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:12.646295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] RBAC: Authorizing `identity:get_user(user_id=271e79eb90be41068da0e18827454fe9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:12.648502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:12.650825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 593/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:12 2022] GET /identity/v3/users/271e79eb90be41068da0e18827454fe9 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:12.658029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:12.769523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9e55f-92ed-494c-81af-75260b8a3f76 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:12.803086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:12.803704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:12.804179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:12.804457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:12.847533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:12.876790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:12.878283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:12.979124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 551/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:27:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:27:12.986102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:13.103204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af78b2-7a4a-4138-acc1-ddcef1c51471 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:13.141762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62985ac-a72c-4569-b852-4ceecde4d4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:13.142396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62985ac-a72c-4569-b852-4ceecde4d4ca None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:13.142663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62985ac-a72c-4569-b852-4ceecde4d4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:13.142917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62985ac-a72c-4569-b852-4ceecde4d4ca None admin] PATH_INFO: `/v3/projects/4a3e06c306a5420098103fc2b0950044` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:13.144097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62985ac-a72c-4569-b852-4ceecde4d4ca None admin] RBAC: Authorizing `identity:get_project(project_id=4a3e06c306a5420098103fc2b0950044)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:13.145548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62985ac-a72c-4569-b852-4ceecde4d4ca None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:13.147213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 594/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:12 2022] GET /identity/v3/projects/4a3e06c306a5420098103fc2b0950044 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:13.153610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:13.223612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:13.258000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86374b13-27b1-461f-a33f-e7fbbabd5b85 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:13.293086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:13.293891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:13.294109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:13.294564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:13.295121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=4a3e06c306a5420098103fc2b0950044, user.id=271e79eb90be41068da0e18827454fe9) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:13.295427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:13.296907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f9367-4f98-40f9-8298-e606efd8f029 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:13.302263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 552/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:27:13 2022] GET /identity/v3/role_assignments?user.id=271e79eb90be41068da0e18827454fe9&scope.project.id=4a3e06c306a5420098103fc2b0950044&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:15.309275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ef78-2ebe-4ccf-bf9a-189cf73ef32b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:15.310100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ef78-2ebe-4ccf-bf9a-189cf73ef32b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:15.310100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ef78-2ebe-4ccf-bf9a-189cf73ef32b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:15.311340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 595/1147] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:15.317659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:15.318124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:15.318213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:15.339851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:15.341383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:15.415847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 553/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:15.439448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ef78-2ebe-4ccf-bf9a-189cf73ef32b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:15.503172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:15.551292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ef78-2ebe-4ccf-bf9a-189cf73ef32b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:15.592359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:15.592912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:15.593178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:15.593445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:15.605160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:27:15.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:15.611215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:27:15.616942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 596/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:27:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:15.619040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:15.694883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:15.736256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365715-3c7f-41cf-a115-36abbb3c6174 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:15.771852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:15.772468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:15.772749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:15.773020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:15.773720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:15.774024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:15.775410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:15.784573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 554/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:27:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:15.790859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:15.903275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d11572-4ebb-4f9a-bacb-252d0bf8e7be None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:15.921637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:15.945197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:15.945839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:15.946114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:15.946387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] PATH_INFO: `/v3/users/271e79eb90be41068da0e18827454fe9` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:15.947798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] RBAC: Authorizing `identity:get_user(user_id=271e79eb90be41068da0e18827454fe9)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:15.950119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:15.951796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 597/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:15 2022] GET /identity/v3/users/271e79eb90be41068da0e18827454fe9 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:15.957774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:16.066999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf8c4-d81f-4802-b584-813c2846459e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:16.098579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:16.108046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdf305-b92b-4c5e-84fa-9e580bfe929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:16.108799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdf305-b92b-4c5e-84fa-9e580bfe929d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:16.109163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdf305-b92b-4c5e-84fa-9e580bfe929d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:16.109526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdf305-b92b-4c5e-84fa-9e580bfe929d None admin] PATH_INFO: `/v3/projects/4a3e06c306a5420098103fc2b0950044` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:16.110920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdf305-b92b-4c5e-84fa-9e580bfe929d None admin] RBAC: Authorizing `identity:get_project(project_id=4a3e06c306a5420098103fc2b0950044)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:16.112554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdf305-b92b-4c5e-84fa-9e580bfe929d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:16.114369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 555/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:15 2022] GET /identity/v3/projects/4a3e06c306a5420098103fc2b0950044 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:16.121101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:16.237227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfba2d-b7d2-4692-a1cc-f5e0dd5ce38d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:16.274234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c2a0-2208-4199-8384-4be208738665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:16.274916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c2a0-2208-4199-8384-4be208738665 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:16.275184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c2a0-2208-4199-8384-4be208738665 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:16.275452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c2a0-2208-4199-8384-4be208738665 None admin] PATH_INFO: `/v3/projects/4a3e06c306a5420098103fc2b0950044/users/271e79eb90be41068da0e18827454fe9/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:16.277445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6c2a0-2208-4199-8384-4be208738665 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a3e06c306a5420098103fc2b0950044, user_id=271e79eb90be41068da0e18827454fe9, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:16.279104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6c2a0-2208-4199-8384-4be208738665 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:16.297676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 598/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:27:16 2022] PUT /identity/v3/projects/4a3e06c306a5420098103fc2b0950044/users/271e79eb90be41068da0e18827454fe9/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:27:18.356416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10ae02-e65d-405f-9eb1-38257621429c None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:18.357032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10ae02-e65d-405f-9eb1-38257621429c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:18.357032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10ae02-e65d-405f-9eb1-38257621429c None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:18.358338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 556/1154] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:18.364072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:18.364546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:18.364637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:18.384357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:18.385533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:18.442428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:18.479737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 599/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:18.503404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10ae02-e65d-405f-9eb1-38257621429c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:18.623689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10ae02-e65d-405f-9eb1-38257621429c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:18.660542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:18.661187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:18.661465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:18.661739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:18.677757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:27:18.685526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:18.686659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:27:18.687707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 557/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:27:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:18.693823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:18.807848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b7aba-ef30-4ddb-bc86-978d63e3cbc8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:18.847101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:18.847726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:18.848040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:18.848305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:18.849026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:18.849325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:18.850708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:18.859994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 600/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:27:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:18.866185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:18.928223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:18.975983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5d724-a744-407e-b4cd-9061edfa95a2 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:19.013910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:19.014550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:19.014828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:19.015101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] PATH_INFO: `/v3/users/271e79eb90be41068da0e18827454fe9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:19.016354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] RBAC: Authorizing `identity:get_user(user_id=271e79eb90be41068da0e18827454fe9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:19.017802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:19.019435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 558/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:18 2022] GET /identity/v3/users/271e79eb90be41068da0e18827454fe9 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:19.026918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:19.068447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:19.144112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa80561-6f40-47e4-b36a-a0e7bea0ff20 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:19.180718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:19.181249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:19.181526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:19.181796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:19.253585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:19.255041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:19.291082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:19.360098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 601/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:27:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:27:19.366252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:19.482963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b4b4d-9c83-4d75-abe9-3dbc664898dc None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:19.507109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:19.519301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b1936-61a2-4a9d-a485-04e136135c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:19.519978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1936-61a2-4a9d-a485-04e136135c7e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:19.520258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1936-61a2-4a9d-a485-04e136135c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:19.520517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1936-61a2-4a9d-a485-04e136135c7e None admin] PATH_INFO: `/v3/projects/4a3e06c306a5420098103fc2b0950044` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:19.521708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b1936-61a2-4a9d-a485-04e136135c7e None admin] RBAC: Authorizing `identity:get_project(project_id=4a3e06c306a5420098103fc2b0950044)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:19.523161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b1936-61a2-4a9d-a485-04e136135c7e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:19.524901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 559/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:19 2022] GET /identity/v3/projects/4a3e06c306a5420098103fc2b0950044 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:19.531883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:19.643610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c96ee-ff4e-4b60-8716-3177feb4d309 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:19.683162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:19.683775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:19.684133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:19.684406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:19.685197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=4a3e06c306a5420098103fc2b0950044, user.id=271e79eb90be41068da0e18827454fe9) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:19.685505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:19.686982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864280b-3c97-43db-8950-e3da10c80e82 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:19.689007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:19.697138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 602/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:27:19 2022] GET /identity/v3/role_assignments?user.id=271e79eb90be41068da0e18827454fe9&scope.project.id=4a3e06c306a5420098103fc2b0950044&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:21.729368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939d398-4afc-4ca4-b9a2-a12f99052ece None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:21.729900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939d398-4afc-4ca4-b9a2-a12f99052ece None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:21.729900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939d398-4afc-4ca4-b9a2-a12f99052ece None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:21.731414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 560/1162] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:21.738731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:21.739239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:21.739640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:21.761406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:21.762740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:21.836615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 603/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:21.860062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939d398-4afc-4ca4-b9a2-a12f99052ece None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:21.965604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939d398-4afc-4ca4-b9a2-a12f99052ece None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:22.001527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:22.002138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:22.002411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:22.002675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:22.061916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:22.072193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:22.073641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:22.180672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 561/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:27:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:27:22.187109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:22.297366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7e003-03c2-4651-8dd9-e615ecd8a321 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:22.324430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:22.333859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:22.334499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:22.334771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:22.335033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:22.341971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:22.343401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-003a6b89-2310-4ad0-9ba5-c58ae4e14912 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:22.354827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:22.356208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:27:22.357302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 604/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:27:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:22.362467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:22.478951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:22.490906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a5425-2d67-4f6f-8b95-05b12eae8664 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:22.527311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:22.528357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:22.528826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:22.529304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:22.530577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:22.531113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:22.533374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cb99f-43fa-4127-be04-82164523c2ec None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:22.540829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 562/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Aug 1 16:27:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:24.511332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e085032-1e83-45c2-97d4-12d9d8fc1158 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:24.512157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e085032-1e83-45c2-97d4-12d9d8fc1158 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:24.512157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e085032-1e83-45c2-97d4-12d9d8fc1158 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:24.513677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 605/1167] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:24.519689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e01fb-ebb6-432c-9209-d29b56074438 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:24.520360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e01fb-ebb6-432c-9209-d29b56074438 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:24.520517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e01fb-ebb6-432c-9209-d29b56074438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:24.548722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a93e01fb-ebb6-432c-9209-d29b56074438 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:24.549947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a93e01fb-ebb6-432c-9209-d29b56074438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:24.625504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 563/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:24.658481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e085032-1e83-45c2-97d4-12d9d8fc1158 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:24.780209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e085032-1e83-45c2-97d4-12d9d8fc1158 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:24.821707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28cdec-480a-4b70-ba21-fa3b5a10c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:24.822300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28cdec-480a-4b70-ba21-fa3b5a10c99c None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:24.822572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28cdec-480a-4b70-ba21-fa3b5a10c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:24.822841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28cdec-480a-4b70-ba21-fa3b5a10c99c None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:24.823617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28cdec-480a-4b70-ba21-fa3b5a10c99c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:24.825082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28cdec-480a-4b70-ba21-fa3b5a10c99c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:24.844081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 606/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Mon Aug 1 16:27:24 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:27:26.934146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabab31-a3c4-40f8-b93f-c3f07ad81e01 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:26.934793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabab31-a3c4-40f8-b93f-c3f07ad81e01 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:26.934793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabab31-a3c4-40f8-b93f-c3f07ad81e01 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:26.935972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 564/1170] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:26.941665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:26.942368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:26.942368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:26.963710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:26.965075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:27.016257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:27.041747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 607/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:27.069755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabab31-a3c4-40f8-b93f-c3f07ad81e01 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:27.170464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:27.187878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabab31-a3c4-40f8-b93f-c3f07ad81e01 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:27.223919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b8c4d-b99a-4fbe-aaaa-aaa02a16139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:27.227265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b8c4d-b99a-4fbe-aaaa-aaa02a16139a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:27.227265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b8c4d-b99a-4fbe-aaaa-aaa02a16139a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:27.227265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b8c4d-b99a-4fbe-aaaa-aaa02a16139a None admin] PATH_INFO: `/v3/domains/da87b4a92df64146b39d102eb4340ef8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:27.236623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b8c4d-b99a-4fbe-aaaa-aaa02a16139a None admin] RBAC: Authorizing `identity:get_domain(domain_id=da87b4a92df64146b39d102eb4340ef8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:27.238081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b8c4d-b99a-4fbe-aaaa-aaa02a16139a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:27.239786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 565/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Mon Aug 1 16:27:27 2022] GET /identity/v3/domains/da87b4a92df64146b39d102eb4340ef8 => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:27.246465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:27.354724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f7b74-9b8b-42f7-b616-4017c331ea3d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:27.390785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a61321-1c83-449a-9d34-3630e463f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:27.391400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a61321-1c83-449a-9d34-3630e463f733 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:27.391671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a61321-1c83-449a-9d34-3630e463f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:27.391966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a61321-1c83-449a-9d34-3630e463f733 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:27.392643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a61321-1c83-449a-9d34-3630e463f733 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:27.394074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a61321-1c83-449a-9d34-3630e463f733 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:27.428156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 608/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:27:27 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:27:29.506176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53ad05-7983-4a6b-bfb9-94aa06c3a46e None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:29.506760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53ad05-7983-4a6b-bfb9-94aa06c3a46e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:29.506760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53ad05-7983-4a6b-bfb9-94aa06c3a46e None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:29.508118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 566/1174] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:29.513928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:29.514388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:29.514501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:29.533908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:29.535143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:29.563078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:29.619552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 609/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:29.647281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53ad05-7983-4a6b-bfb9-94aa06c3a46e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:29.759927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53ad05-7983-4a6b-bfb9-94aa06c3a46e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:29.795791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4d181-e1e0-417e-9ff7-ee595ea9143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:29.796485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4d181-e1e0-417e-9ff7-ee595ea9143e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:29.796765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4d181-e1e0-417e-9ff7-ee595ea9143e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:29.797040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4d181-e1e0-417e-9ff7-ee595ea9143e None admin] PATH_INFO: `/v3/domains/da87b4a92df64146b39d102eb4340ef8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:29.798325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4d181-e1e0-417e-9ff7-ee595ea9143e None admin] RBAC: Authorizing `identity:get_domain(domain_id=da87b4a92df64146b39d102eb4340ef8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:29.799754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4d181-e1e0-417e-9ff7-ee595ea9143e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:29.801397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 567/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Mon Aug 1 16:27:29 2022] GET /identity/v3/domains/da87b4a92df64146b39d102eb4340ef8 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:29.811454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:29.928127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383c7a-c5ff-4549-99a2-37f175135115 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:29.966860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249450e-5a22-4c5b-9272-98c3de32a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:29.967487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249450e-5a22-4c5b-9272-98c3de32a94a None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:29.967790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249450e-5a22-4c5b-9272-98c3de32a94a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:29.968060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249450e-5a22-4c5b-9272-98c3de32a94a None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:29.968750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249450e-5a22-4c5b-9272-98c3de32a94a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:29.970238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249450e-5a22-4c5b-9272-98c3de32a94a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:30.000965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 610/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:27:29 2022] POST /identity/v3/users => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:27:32.090843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981704f-9f6d-49e4-9df6-687760109515 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:32.091396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981704f-9f6d-49e4-9df6-687760109515 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:32.091396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981704f-9f6d-49e4-9df6-687760109515 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:32.092781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 568/1178] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:32.098007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:32.098800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:32.099081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:32.117442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:32.118628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:32.131073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:32.186414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 611/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:27:32.209853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981704f-9f6d-49e4-9df6-687760109515 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:32.227403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:32.314799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981704f-9f6d-49e4-9df6-687760109515 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:32.349086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:32.349642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:32.349916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:32.350184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:32.361175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:27:32.365984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:32.367071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:27:32.368134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 569/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:27:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:32.374167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:32.476199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad85a7-6ffb-47ee-8a45-746153ddc6bd None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:32.511143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:32.511926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:32.512200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:32.512469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:32.513183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:32.513484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:32.514865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:32.523079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 612/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:27:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:32.529316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:32.608109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:32.636115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664a94e-aaff-4231-b84a-e4d7ba32ead9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:32.671043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:32.671679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:32.671976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:32.672228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] PATH_INFO: `/v3/users/f3ebb121c5a34a60a9f087988eaf8f6b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:32.686737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] RBAC: Authorizing `identity:get_user(user_id=f3ebb121c5a34a60a9f087988eaf8f6b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:32.688218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:32.689870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 570/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:32 2022] GET /identity/v3/users/f3ebb121c5a34a60a9f087988eaf8f6b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:32.695909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:32.708977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:32.802236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120c6d-b8f4-4d33-bd38-fd2fb34017c0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:32.837329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:32.837952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:32.838229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:32.838503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:32.910276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:32.911652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:32.921226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:33.015084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 613/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:27:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:27:33.021224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:33.124878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867788-89dc-446b-8171-4e8d483aa33b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:33.155802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:33.161991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9b822-d22b-4d36-afd5-c21140d54a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:33.162556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9b822-d22b-4d36-afd5-c21140d54a23 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:33.162832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9b822-d22b-4d36-afd5-c21140d54a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:33.163100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9b822-d22b-4d36-afd5-c21140d54a23 None admin] PATH_INFO: `/v3/projects/837291a6c87740489e7de04919db18dc` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:33.164312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9b822-d22b-4d36-afd5-c21140d54a23 None admin] RBAC: Authorizing `identity:get_project(project_id=837291a6c87740489e7de04919db18dc)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:33.165753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9b822-d22b-4d36-afd5-c21140d54a23 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:33.167406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 571/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:33 2022] GET /identity/v3/projects/837291a6c87740489e7de04919db18dc => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:33.174045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:33.279264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01560a-da66-4e4e-80c8-df7d52afae4f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:33.288504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:33.316868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:33.317474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:33.317749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:33.318021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:33.318799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=837291a6c87740489e7de04919db18dc, user.id=f3ebb121c5a34a60a9f087988eaf8f6b) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:33.319103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:33.320592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123efbd8-dc61-4027-8365-f91665082e96 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:33.326355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 614/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:27:33 2022] GET /identity/v3/role_assignments?user.id=f3ebb121c5a34a60a9f087988eaf8f6b&scope.project.id=837291a6c87740489e7de04919db18dc&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:35.284235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49818-dc83-4d7b-b1ce-df41960485e3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:35.284842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49818-dc83-4d7b-b1ce-df41960485e3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:35.284842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49818-dc83-4d7b-b1ce-df41960485e3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:35.286105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 572/1186] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:35.291499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:35.291961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:35.292061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:35.310663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:35.311885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:35.380169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 615/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:35.402798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab49818-dc83-4d7b-b1ce-df41960485e3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:35.509423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab49818-dc83-4d7b-b1ce-df41960485e3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:35.545996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:35.546564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:35.546846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:35.547111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:35.557575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:27:35.562664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:35.563830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:27:35.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 573/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:27:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:35.570156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:35.670211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd199d-4eaa-4e58-8769-d7b40b38beba None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:35.671852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:35.707428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:35.708073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:35.708335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:35.708606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:35.709319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:35.709614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:35.710984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:35.719368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 616/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:27:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:35.725435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:35.786999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:35.828998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57045a0-3401-4b53-b141-a6e9971ef33b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:35.867751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:35.868409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:35.868685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:35.869270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] PATH_INFO: `/v3/users/f3ebb121c5a34a60a9f087988eaf8f6b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:35.870472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] RBAC: Authorizing `identity:get_user(user_id=f3ebb121c5a34a60a9f087988eaf8f6b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:35.872096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:35.873725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 574/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:35 2022] GET /identity/v3/users/f3ebb121c5a34a60a9f087988eaf8f6b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:35.879781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:35.983094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3ceff-50c5-4936-aeb9-889d74dc4cb5 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:36.017787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257340ac-6781-4772-8f43-41153947fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:36.018415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257340ac-6781-4772-8f43-41153947fb18 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:36.018690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257340ac-6781-4772-8f43-41153947fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:36.018957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257340ac-6781-4772-8f43-41153947fb18 None admin] PATH_INFO: `/v3/projects/837291a6c87740489e7de04919db18dc` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:36.020189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257340ac-6781-4772-8f43-41153947fb18 None admin] RBAC: Authorizing `identity:get_project(project_id=837291a6c87740489e7de04919db18dc)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:36.021625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257340ac-6781-4772-8f43-41153947fb18 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:36.023300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 617/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:35 2022] GET /identity/v3/projects/837291a6c87740489e7de04919db18dc => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:36.029699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:36.130992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6b11b-2021-4c3a-839e-71a2c3c3f3ba None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:36.157850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:36.166867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89793caf-8463-43f3-a98b-bf9b30456640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:36.167580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793caf-8463-43f3-a98b-bf9b30456640 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:36.168026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793caf-8463-43f3-a98b-bf9b30456640 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:36.168289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89793caf-8463-43f3-a98b-bf9b30456640 None admin] PATH_INFO: `/v3/projects/837291a6c87740489e7de04919db18dc/users/f3ebb121c5a34a60a9f087988eaf8f6b/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:36.170210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793caf-8463-43f3-a98b-bf9b30456640 None admin] RBAC: Authorizing `identity:create_grant(project_id=837291a6c87740489e7de04919db18dc, user_id=f3ebb121c5a34a60a9f087988eaf8f6b, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:36.171643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89793caf-8463-43f3-a98b-bf9b30456640 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:36.186147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 575/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:27:36 2022] PUT /identity/v3/projects/837291a6c87740489e7de04919db18dc/users/f3ebb121c5a34a60a9f087988eaf8f6b/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:27:38.306652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73fce0-6c71-4f2c-8d5e-4748777b6350 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:38.306652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73fce0-6c71-4f2c-8d5e-4748777b6350 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:38.306652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73fce0-6c71-4f2c-8d5e-4748777b6350 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:38.309436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 618/1193] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:27:38.315225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:38.315717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:38.315832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:38.336518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:27:38.337736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:27:38.412485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 576/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:27:38.443594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73fce0-6c71-4f2c-8d5e-4748777b6350 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:38.477679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:38.566945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73fce0-6c71-4f2c-8d5e-4748777b6350 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:38.604591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:38.605990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:38.606282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:38.606560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:38.617319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:27:38.623276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:27:38.624559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:27:38.625662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 619/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:27:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:27:38.630911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:38.745471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec32047-f9e6-4540-8eb6-96142ab68048 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:38.771831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:38.791881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:38.792818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:38.793094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:38.793363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:38.794088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:38.794390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:38.795802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:38.805463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 577/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:27:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:38.812742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:38.927454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:38.937589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c048e6-c33a-4552-9af8-4176299c760f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:38.977018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:38.977667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:38.977943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:38.978220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] PATH_INFO: `/v3/users/f3ebb121c5a34a60a9f087988eaf8f6b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:38.979430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] RBAC: Authorizing `identity:get_user(user_id=f3ebb121c5a34a60a9f087988eaf8f6b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:38.981326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:38.983007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 620/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Aug 1 16:27:38 2022] GET /identity/v3/users/f3ebb121c5a34a60a9f087988eaf8f6b => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:38.989030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:39.106165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627a111-33bd-4513-99df-848b4c236341 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:39.144458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:39.145091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:39.145369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:39.145640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:39.196025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:39.221598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:39.223106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:39.330959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 578/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:27:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:27:39.337275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:39.446800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d0a0c-a439-4f45-8905-1a69daf1f5c1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:39.482727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f4466-afcd-44a1-8b21-487012092335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:39.483401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4466-afcd-44a1-8b21-487012092335 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:39.483674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4466-afcd-44a1-8b21-487012092335 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:39.483967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f4466-afcd-44a1-8b21-487012092335 None admin] PATH_INFO: `/v3/projects/837291a6c87740489e7de04919db18dc` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:39.485163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4466-afcd-44a1-8b21-487012092335 None admin] RBAC: Authorizing `identity:get_project(project_id=837291a6c87740489e7de04919db18dc)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:39.486600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f4466-afcd-44a1-8b21-487012092335 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:39.488349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 621/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Aug 1 16:27:39 2022] GET /identity/v3/projects/837291a6c87740489e7de04919db18dc => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:39.495199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:27:39.585699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:39.606853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b68aa-5414-48b1-be5f-d415675f07ae None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:27:39.645291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:27:39.645989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:27:39.646270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:27:39.646536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:27:39.647328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=837291a6c87740489e7de04919db18dc, user.id=f3ebb121c5a34a60a9f087988eaf8f6b) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:27:39.647634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:27:39.649110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01fea-1baa-46d1-aa19-7950e700ec36 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:27:39.656327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 579/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:27:39 2022] GET /identity/v3/role_assignments?user.id=f3ebb121c5a34a60a9f087988eaf8f6b&scope.project.id=837291a6c87740489e7de04919db18dc&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:27:49.478193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=86 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:27:49.653238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=28 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:36.142546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f43557-ec1e-4c7e-b050-4783a2ab1c5b None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:36.143192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f43557-ec1e-4c7e-b050-4783a2ab1c5b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:36.143192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f43557-ec1e-4c7e-b050-4783a2ab1c5b None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:36.144463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 622/1201] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:28:36.151814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:36.151814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:36.152869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:36.174242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:28:36.175570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:28:36.275921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 580/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:28:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:28:36.300841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f43557-ec1e-4c7e-b050-4783a2ab1c5b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:36.449221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f43557-ec1e-4c7e-b050-4783a2ab1c5b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:36.498683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:36.499304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:36.499594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:36.499880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:36.512661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:36.514280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:36.535051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:36.536183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:36.537168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:36.537168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:36.537168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:36.537168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:36.537168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 623/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:36.543573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:36.692495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae498-d19a-4ad0-9b87-bf8e889996ae None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:36.729255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:36.738129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:36.738797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:36.739190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:36.739530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:36.740525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:36.741219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:36.743103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a2d42-2eeb-4932-a001-372b3a8f4019 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:36.757983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 581/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:36.763979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:36.835797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:36.896611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f331-f0ec-4975-80d6-80ddb37a81e9 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:36.943017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf757912-02df-4586-bb1c-2b1bb28e4429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:36.943971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf757912-02df-4586-bb1c-2b1bb28e4429 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:36.944403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf757912-02df-4586-bb1c-2b1bb28e4429 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:36.944810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf757912-02df-4586-bb1c-2b1bb28e4429 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:36.946288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf757912-02df-4586-bb1c-2b1bb28e4429 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:36.948522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf757912-02df-4586-bb1c-2b1bb28e4429 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:36.972486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 624/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Aug 1 16:28:36 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:28:39.226855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebc5d3-b07f-42ce-951f-3fb0d236cde8 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:39.227442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebc5d3-b07f-42ce-951f-3fb0d236cde8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:39.227442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebc5d3-b07f-42ce-951f-3fb0d236cde8 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:39.229249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 582/1206] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:28:39.235224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:39.235659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:39.235833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:39.258205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:28:39.259516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:28:39.333648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 625/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:28:39.357718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebc5d3-b07f-42ce-951f-3fb0d236cde8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:39.498283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebc5d3-b07f-42ce-951f-3fb0d236cde8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:39.551978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:39.552637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:39.552914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:39.553178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:39.570573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:28:39.584906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:39.586042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:28:39.586974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:39.591634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 583/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:39.593392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:39.661440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:39.739682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4dd7b-a2d8-42e1-98cc-0e0cd69b45db None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:39.785167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:39.791797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:39.791797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:39.791797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:39.791797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:39.791797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:39.791797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:39.800780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 626/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:28:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:39.806793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:39.863822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:39.934607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993029f6-d0ac-4f95-81f1-e53848af091c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:39.983528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:39.984351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:39.984668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:39.984980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:39.999387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:40.000954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:40.018913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:40.020095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:40.021042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:40.021042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:40.021042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:40.021042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:40.021042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 584/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:40.026732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:40.151449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:40.155194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007324b8-85f9-4a8e-ae56-75aa39e0e413 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:40.195693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:40.196363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:40.196742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:40.197016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:40.197724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:40.198028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:40.199435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:40.216068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 627/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:40.222592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:40.338502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660529c-a3ed-4a8d-860b-3187925c363d None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:40.340512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:40.386151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:40.389173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:40.389173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:40.389173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:40.396955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:40.399238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:28:40.412475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:40.413663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:40.415081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 585/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:40.420530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:40.537232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ab9e-ea9c-4928-bc63-6a6548a377fa None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:40.577855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:40.578519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:40.578796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:40.579116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:40.580032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:40.581544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:40.582057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:40.610918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 628/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 16:28:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:40.617958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:40.740756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5861-c779-444b-9cb2-bdea84405484 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:40.780914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:40.781599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:40.781842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:40.782116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:40.816183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:40.858943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:40.861681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:40.991418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 586/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:28:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:28:40.996701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:41.017448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:41.122012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96884e-4629-4a99-b755-5fb3b0dfa018 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:41.158166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:41.159123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:41.159372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:41.159562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:41.172743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:41.174373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:41.192732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:41.193765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:41.203301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:41.203301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:41.203301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:41.203301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:41.203301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 629/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:41.203672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:41.282048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:41.346714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b46678-bf0e-4126-a356-ec703e7a8d95 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:41.395730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:41.395730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:41.395730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:41.395730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:41.397170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:41.397525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:41.399528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:41.420332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 587/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:41.422297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:41.511274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:41.552932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff7f29-21c3-4a41-91dc-46c2224a81de None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:41.598854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:41.599563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:41.599947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:41.600256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:41.609622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:41.611494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:28:41.630135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:41.631909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:41.633923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:41.636071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 630/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:28:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:41.642445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:41.782703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446577d7-7a0a-4a17-96b7-54bc96309656 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:41.807247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:41.820340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:41.821203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:41.821549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:41.821885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:41.822710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:41.823074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:41.824602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05774ca-9bf2-46ec-bccd-bb5d99b0ec5b None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:41.838650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 588/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:41.847009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:41.975227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a3088-5c2a-43a4-bb7f-41a06a0789f8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:42.012986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:42.021303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:42.022068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:42.022349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:42.022624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:42.023576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=53482e2f81364526ae2686f42a7d2483) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:42.023974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:42.025449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4fe20-f5ae-4cb9-be18-073e2c62bc02 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:42.031862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 631/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:28:41 2022] GET /identity/v3/role_assignments?user.id=53482e2f81364526ae2686f42a7d2483&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:28:44.059381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ae65f-9825-405b-8693-9140118872bf None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:44.060020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ae65f-9825-405b-8693-9140118872bf None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:44.060020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ae65f-9825-405b-8693-9140118872bf None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:44.061329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 589/1220] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:28:44.066990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:44.067486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:44.067599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:44.088172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:28:44.089364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:28:44.164837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 632/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:28:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:28:44.190042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ae65f-9825-405b-8693-9140118872bf None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:44.317220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ae65f-9825-405b-8693-9140118872bf None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:44.359690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:44.360702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:44.361215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:44.361649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:44.376130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:28:44.381197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:44.382261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:44.383227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 590/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:28:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:44.389098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:44.498467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ae750-849a-4f68-86b7-d55e75c431aa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:44.524681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:44.537768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:44.538404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:44.538682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:44.539193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:44.540310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] RBAC: Adding query filter params (name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:44.540616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:44.542084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:44.551892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 633/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:44.558749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:44.614182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:44.685345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4af79-4ccd-4d2d-ba84-fa8792d3d90b None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:44.720910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:44.721554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:44.721830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:44.722096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:44.732319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:44.733821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:44.751001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:44.752061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:44.758545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:44.758545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:44.758545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:44.758545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:44.758545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 591/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:44.758950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:44.873911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6deb72-28da-47e3-8bb6-f7273accb92c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:44.914509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:44.915289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:44.915570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:44.915870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:44.916575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:44.916928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:44.918741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:44.939288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 634/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:44.940781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:45.054986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:45.064510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3f7b7-c0f3-40da-a00c-fade7905e27f None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:45.102408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:45.103104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:45.103410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:45.103716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:45.112548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:45.114046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:28:45.127715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:45.129001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:45.130211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 592/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:45.134939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:45.188401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:45.250388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050487bd-68d1-4160-9706-7e45ab4b169f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:45.289547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:45.290167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:45.290445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:45.290715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:45.291524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:45.291863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:45.293313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:45.311835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 635/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 16:28:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:45.317915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:45.440734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ecb8b-1117-4d92-9e1b-840eadaae37a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:45.480758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:45.481524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:45.481855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:45.482192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:45.497391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:45.498873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:45.513730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:45.514892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:45.516103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:45.516103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:45.516103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:45.516103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:45.516103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 593/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:28:45.521193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:45.649361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e6041-fc30-4643-940a-053cacd86a8c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:45.652040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:45.697982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:45.698531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:45.698804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:45.699090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:45.699807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:45.700104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:45.701481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:45.713683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 636/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:45.719889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:45.732184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:45.836025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847ffaa-b8ae-4fe4-8540-0ffcc3714a93 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:45.870920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:45.871600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:45.871901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:45.872172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:45.879214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:45.880702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:28:45.893211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:45.894392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:45.895528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 594/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:28:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:45.900673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:46.029185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d763-c6c4-4a72-a703-891c2eef66b2 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:46.066992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:46.067616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:46.067924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:46.068194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:46.068942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:46.069248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:46.070779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1a1a7-c5b3-4826-b919-1b93bfc04ccf None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:46.084906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 637/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:28:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:46.091472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:46.163239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:46.202092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba18ba2-436f-4829-950f-48c12298d0a8 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:46.238481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c2726-152d-4831-9725-9a770e619338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:46.239253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c2726-152d-4831-9725-9a770e619338 None admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:46.239505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c2726-152d-4831-9725-9a770e619338 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:46.239792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c2726-152d-4831-9725-9a770e619338 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/53482e2f81364526ae2686f42a7d2483/roles/a264fb96430544f094c1851e7e086dc7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:46.255542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c2726-152d-4831-9725-9a770e619338 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=53482e2f81364526ae2686f42a7d2483, role_id=a264fb96430544f094c1851e7e086dc7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:46.257041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c2726-152d-4831-9725-9a770e619338 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:46.272071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 595/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:28:46 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/53482e2f81364526ae2686f42a7d2483/roles/a264fb96430544f094c1851e7e086dc7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:28:48.479514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eba7ca-a61b-4b91-9bee-5a54deddb733 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:48.480151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eba7ca-a61b-4b91-9bee-5a54deddb733 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:48.480151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eba7ca-a61b-4b91-9bee-5a54deddb733 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:48.481706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 638/1233] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:28:48.487191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:48.487704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:48.487838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:48.507637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:28:48.509084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:28:48.583551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 596/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:28:48.608715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eba7ca-a61b-4b91-9bee-5a54deddb733 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:48.637722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:48.730250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eba7ca-a61b-4b91-9bee-5a54deddb733 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:48.772401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:48.773149 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:48.773434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:48.775209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:48.785791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:28:48.792660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:48.794037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:48.795404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 639/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Aug 1 16:28:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:48.802367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:48.854084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:48.917226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f49970-4af6-49cc-8eba-3e2e0706953a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:48.954690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:48.955320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:48.955597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:48.955895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:48.956614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:48.956916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:48.958322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:48.967719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 597/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Aug 1 16:28:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:48.973751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:49.071397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:49.089204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6f695-a982-4436-bd87-90b4ce51123d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:49.127382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:49.128185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:49.128502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:49.128905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:49.141503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:49.142992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:49.159151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:49.160363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:49.161470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:49.161470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:49.161470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:49.161470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:49.161470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 640/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:49.167443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:49.297257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e20f9-2c17-432f-b328-53fa39b01c59 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:49.307403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:49.336775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:49.337332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:49.337611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:49.337984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:49.338576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:49.338880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:49.340333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:49.354069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 598/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:49.360234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:49.466812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469cf7-fe56-4dd7-9c81-1985d2a1dc2b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:49.502871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:49.503607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:49.503825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:49.504073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:49.506719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:49.512967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:49.514431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:28:49.526434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:49.527666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:49.528840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 641/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:49.533615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:49.650087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8579d-50a6-461c-84a5-75e3763e70be None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:49.694162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:49.695021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:49.695325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:49.695639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:49.696455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:49.696755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:49.698195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:49.717027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 599/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 16:28:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:49.723196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:49.832605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e541-4142-4df0-b799-9733b5f0772a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:49.871384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:49.872052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:49.872326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:49.872592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:49.935286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:49.943467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:49.945272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:50.053932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 642/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:28:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:28:50.060314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:50.092683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:50.175621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb122d-eb5a-460e-bf61-6a8f1ee7c382 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:50.217344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:50.217948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:50.218197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:50.218468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:50.234116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:50.235625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:50.253961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:50.255108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:50.256490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:50.256490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:50.256490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:50.256490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:50.256490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 600/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:50.262101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:50.373275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:50.380304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518e90-045b-41e2-b21d-f013dda43077 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:50.418931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:50.419594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:50.419897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:50.420193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:50.420947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:50.421237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:50.422661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:50.435206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 643/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:28:50.441296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:50.533122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:50.558451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4a4d7-75a2-40d8-8ce4-261bb2b0d3b0 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:50.596502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:50.597410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:50.597727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:50.598114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:50.605415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:50.606899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:28:50.620005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:50.621180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:50.622344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 601/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:28:50.627985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:50.743563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc8ce7-3e73-4f57-a437-582bb6623338 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:50.794092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:50.794640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:50.794922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:50.795190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:50.795967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:50.796275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:50.797713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6a39-bcb1-47b5-b544-ee7dc32a12fb None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:50.811705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 644/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:50.818223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:50.940599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572450f-593c-490d-9ebe-4f5cd4ebd473 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:50.985026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:50.985652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:50.985921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:50.986202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:50.987212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] RBAC: Adding query filter params (role.id=a264fb96430544f094c1851e7e086dc7, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=53482e2f81364526ae2686f42a7d2483) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:50.987512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:50.989069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714e2fe-4a97-410f-827f-e2803c9deb4c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:50.996467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 602/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:28:50 2022] GET /identity/v3/role_assignments?user.id=53482e2f81364526ae2686f42a7d2483&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=a264fb96430544f094c1851e7e086dc7 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:53.292595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49210f5-9850-445d-a9de-282cf2a31187 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:53.293208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49210f5-9850-445d-a9de-282cf2a31187 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:53.293307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49210f5-9850-445d-a9de-282cf2a31187 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:53.294953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 645/1247] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:28:53.300579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:53.301070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:53.301265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:53.309214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:53.323583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:28:53.324858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:28:53.403353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 603/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:28:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:28:53.426490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49210f5-9850-445d-a9de-282cf2a31187 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:53.446032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:53.544244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49210f5-9850-445d-a9de-282cf2a31187 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:53.583077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:53.583750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:53.584200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:53.584475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:53.599316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:28:53.604784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:53.605813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:28:53.606741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:53.611275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 646/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:53.612316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:53.735848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6030f3-cfd1-4510-ae1d-fe4ddc87175e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:53.769848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:53.775992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:53.776641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:53.776964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:53.777325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:53.778123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:53.778422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:53.779894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:53.789534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 604/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:28:53.796110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:53.884401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:53.913193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cacbf-0c41-43e6-a836-9edbbb763f63 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:53.954717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:53.955547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:53.955946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:53.956290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:53.971945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:53.973959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:53.998535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:53.999839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:54.000988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:54.000988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:54.000988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:54.000988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:54.000988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 647/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:54.007027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:54.131508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994737-1519-4f48-b5a4-dbe6ea5ff8b9 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:54.170554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:54.171243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:54.171531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:54.171836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:54.172546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:54.172856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:54.174266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:54.188652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 605/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:54.195229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:54.313677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7ebd-3c67-4257-b3df-a76a3a54ca90 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:54.345890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:54.356030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:54.356705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:54.356964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:54.357233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:54.367675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:54.369569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:28:54.383914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:54.385340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:54.386772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 648/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:28:54.391677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:54.457845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:54.512163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e462f-b205-4253-bd6e-4a3028c7ce68 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:54.550817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:54.551525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:54.551850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:54.552209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:54.553091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:54.553516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:54.555335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:54.578674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 606/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 16:28:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:54.584696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:54.708394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec315f-4a9a-4906-90f1-a67c01a4d1e8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:54.749323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:54.750132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:54.750365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:54.750659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:54.814847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:54.829372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:54.831021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:54.953979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 649/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:28:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:28:54.960585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:55.068425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c837da-77d9-48e9-9604-081dcf62c9d5 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:55.083606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:55.108097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:55.108780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:55.109049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:55.109316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:55.121053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:55.122506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:55.137896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:55.139307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:55.140791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:55.140791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:55.140791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:55.140791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:55.140791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 607/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:55.145164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:55.251969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:55.267961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f1cd7-66a4-44c6-a297-8b479d8f393f None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:55.309379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:55.310106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:55.310385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:55.310696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:55.311516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:55.311854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:55.313481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:55.329405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 650/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:55.335320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:55.454101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fdb13-b5bb-4e5c-aae5-86f7dd1f39ce None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:55.495424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:55.496426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:55.496703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:55.496970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:55.504812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:55.506467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:28:55.521555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:55.522761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:28:55.523888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 608/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:55.529315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:55.645646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22262001-122e-4bde-9db7-c217c492c30d None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:55.687807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:55.688443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:55.688719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:55.688985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:55.689811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:55.690208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:55.691783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bb4be-31ca-4e8f-b003-ee3f2c346af0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:55.712238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 651/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:55.720559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:55.764043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:55.863481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2049b-2172-4dc3-ace0-ecb05d40b783 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:55.904603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:55.905401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:55.905742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:55.906005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:55.906905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=53482e2f81364526ae2686f42a7d2483) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:55.907261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:55.909012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5e5b6-5a18-4787-b561-02ba975be77f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:55.915841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 609/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:28:55 2022] GET /identity/v3/role_assignments?user.id=53482e2f81364526ae2686f42a7d2483&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:28:58.095820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efda54-f059-45fe-8d0b-f805a75d9600 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:58.096414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efda54-f059-45fe-8d0b-f805a75d9600 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:58.096414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efda54-f059-45fe-8d0b-f805a75d9600 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:58.097989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 652/1261] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:28:58.104084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:58.104615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:58.104615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:58.125758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:28:58.127151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:28:58.205878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 610/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:28:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:28:58.230589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efda54-f059-45fe-8d0b-f805a75d9600 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:58.245192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:58.380038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efda54-f059-45fe-8d0b-f805a75d9600 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:58.428452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:58.429188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:58.429507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:58.429836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:58.443653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:28:58.448968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:58.450080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:58.451202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 653/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:58.456764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:58.482566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:58.578483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618820-dcc2-4429-bf89-e7aa7c46e77e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:58.620799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:58.621438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:58.621770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:58.622042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:58.622764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:58.623068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:58.624490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:58.634273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 611/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:58.640294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:58.714334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:58.749182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea74d07-a97c-4033-b475-809b02aad5d1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:58.783357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:58.784037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:58.784311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:58.784583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:58.795127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:58.796659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:58.811996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:58.813126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:58.814061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:58.814061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:58.814061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:58.814061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:58.814061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 654/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:58.820220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:58.915127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:58.937120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51cd11-da50-4d3f-bb1b-453da8542a04 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:58.977214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:58.977842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:58.978124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:58.978390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:58.979080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:58.979380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:58.980813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:58.993603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 612/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:58.999718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:59.108495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc4c6a-5c9d-47b3-8f3b-201c358a0e96 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:59.129194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:59.144634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:59.145282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:59.145557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:59.145830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:59.154262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:59.155725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:28:59.167447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:59.168562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:59.169847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 655/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:59.174611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:59.281446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14be845-85e7-47ab-8e9b-71abac19474a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:59.317659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:59.318275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:59.318548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:59.318813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:59.319875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:59.320184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:59.321626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:59.331239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:59.341409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 613/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 16:28:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:59.347561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:59.464526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d52623-7aa8-456e-aca4-a502fad55909 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:59.500900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:59.501548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:59.501823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:59.502093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:59.514291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:59.515795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:28:59.530999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:59.532145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:59.533071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:28:59.533071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:28:59.533071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:28:59.533071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:59.533071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 656/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:59.538582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:59.648825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618a062-d343-4605-8263-c76e0f366cca None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:59.705479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:59.706118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:59.706397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:59.706666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:59.707363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:28:59.707667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:59.709115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:59.721802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 614/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:28:59.728773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:59.785829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:28:59.854473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1478f4-1c97-4d37-ad12-bdddb033de3a None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:28:59.889946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:28:59.890578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:28:59.890853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:28:59.891123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:28:59.898139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:28:59.899586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:28:59.911104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:28:59.912143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:28:59.913300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 657/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:28:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:28:59.918915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:28:59.989014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:00.038284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af225784-5cc9-4d90-ab15-232e86d1e002 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:00.074894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:00.075492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:00.075778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:00.076063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:00.076813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:00.077122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:00.078696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0bf434-c154-4c63-bc6e-2b78291562cc None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:00.092232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 615/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:28:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:00.099599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:00.196207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:00.206782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08470de9-7f2b-4ba0-b5d8-fb9f4b328444 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:00.250241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f3495-2825-4f6d-a8b8-84b7ca48b9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:00.250907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f3495-2825-4f6d-a8b8-84b7ca48b9b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:00.251189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f3495-2825-4f6d-a8b8-84b7ca48b9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:00.251461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f3495-2825-4f6d-a8b8-84b7ca48b9b8 None admin] PATH_INFO: `/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/53482e2f81364526ae2686f42a7d2483/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:00.253929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f3495-2825-4f6d-a8b8-84b7ca48b9b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6264f1e9c3c649e49be50ca0b0f66888, user_id=53482e2f81364526ae2686f42a7d2483, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:00.255746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f3495-2825-4f6d-a8b8-84b7ca48b9b8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:00.273207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 658/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Aug 1 16:29:00 2022] PUT /identity/v3/projects/6264f1e9c3c649e49be50ca0b0f66888/users/53482e2f81364526ae2686f42a7d2483/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:29:02.326839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009e61-b1d1-4e07-bc14-331ceb0aa21a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:02.327541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009e61-b1d1-4e07-bc14-331ceb0aa21a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:02.327541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009e61-b1d1-4e07-bc14-331ceb0aa21a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:02.329275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 616/1274] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:29:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:29:02.337576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:02.339594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:02.339594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:02.361840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:29:02.363124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:29:02.467304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 659/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:29:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:29:02.496103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97009e61-b1d1-4e07-bc14-331ceb0aa21a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:02.632454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97009e61-b1d1-4e07-bc14-331ceb0aa21a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:02.652445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:02.673403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:02.674061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:02.675636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:02.675918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:02.691878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:29:02.698545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:02.699964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:02.705207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 617/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Aug 1 16:29:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:02.706451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:02.809181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:02.832924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6e939-2779-4e4c-9920-03332e093f7c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:02.870382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:02.871004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:02.871276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:02.871537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:02.872281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:02.872575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:02.873950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:02.882572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 660/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Aug 1 16:29:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:02.888823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:03.005481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c2e8-7a65-46b4-b791-540a567dd010 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:03.041908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:03.042544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:03.042815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:03.043078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:03.055602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:03.057116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:29:03.074419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:03.075460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:03.076580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:29:03.076580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:29:03.076580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:29:03.076580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:03.076580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 618/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:03.081883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:03.188509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce16da1-5b31-4160-80bc-cb1c208d67f1 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:03.224460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:03.225077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:03.225747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:03.226080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:03.226772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:03.227077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:03.228490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:03.241841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 661/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:03.251260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:03.294310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:03.375996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812978e-0246-40ec-973b-74d79729cbaf None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:03.411869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:03.412508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:03.412838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:03.413316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:03.427008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:03.428561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:29:03.441007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:03.442042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:03.445409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 619/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:29:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:03.450890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:03.455216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:03.559617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570813f-2d81-41da-a681-855e0b3f29a8 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:03.595549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:03.596194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:03.596468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:03.596738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:03.597524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:03.597826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:03.599233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:03.616792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 662/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Aug 1 16:29:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:29:03.622871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:03.724594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:03.738605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b2c6a-9749-4ae6-8c90-d9d6aeded371 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:03.774608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:03.775223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:03.775496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:03.775797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:03.869611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:03.871082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:03.968805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:03.980607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 620/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Aug 1 16:29:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:29:03.989713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:04.052641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:04.096186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4614-4f9d-42bb-9e19-3bf47023a02b None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:04.134134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:04.134773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:04.135047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:04.135313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:04.145783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:04.147265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:29:04.162067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:04.163178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:04.164273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:29:04.164273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:29:04.164273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:29:04.164273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:29:04.164273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 663/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:04.171002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:04.279594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117b114-05a8-4bae-8bf1-0a2444ea570c None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:04.315333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:04.315977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:04.316256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:04.316529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:04.317218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:04.317522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:04.318915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:04.330838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 621/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Aug 1 16:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:04.339126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:04.450275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a5a22-2dd4-4d85-ab1c-4c213d36468c None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:04.465411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:04.489464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:04.490131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:04.490401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:04.490667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:04.497707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:04.499144 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:29:04.510623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:04.511653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: ERROR keystone.server.flask.application Aug 01 16:29:04.512765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 664/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Aug 1 16:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:04.517895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:04.575431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:04.629945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b9d90-363e-4463-9357-e68e6a2c4d3e None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:04.665708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:04.666331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:04.666609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:04.666882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:04.667633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:04.667981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:04.669431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465e6d7-2e52-4552-b998-71cdb19b55ce None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:04.682878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 622/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Aug 1 16:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:04.689256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:04.801260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ed230-ab5b-43d9-9c47-15ac81205e62 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:04.838221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:04.838747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:04.839016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:04.839279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:04.840093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] RBAC: Adding query filter params (role.id=bdcad476ea6d4b5f9facf826c9c4b296, scope.project.id=6264f1e9c3c649e49be50ca0b0f66888, user.id=53482e2f81364526ae2686f42a7d2483) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:04.840392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:04.841811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ccc02-7961-4410-9da6-aaaca68edac1 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:04.849845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 665/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Aug 1 16:29:04 2022] GET /identity/v3/role_assignments?user.id=53482e2f81364526ae2686f42a7d2483&scope.project.id=6264f1e9c3c649e49be50ca0b0f66888&role.id=bdcad476ea6d4b5f9facf826c9c4b296 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:06.904379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68acca4-3165-420d-999e-51468e4b7823 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:06.904937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68acca4-3165-420d-999e-51468e4b7823 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:06.904937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68acca4-3165-420d-999e-51468e4b7823 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:06.906278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 623/1288] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:29:06.911744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:06.912208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:06.912300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:06.931854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:29:06.932976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:29:06.964122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:07.002836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 666/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:29:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:29:07.024779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68acca4-3165-420d-999e-51468e4b7823 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:07.134427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68acca4-3165-420d-999e-51468e4b7823 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:07.145212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:07.170497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:07.171144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:07.171428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:07.171700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] PATH_INFO: `/v3/services/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:07.172361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:07.173736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:07.183084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:07.187991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:07.189105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 624/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:29:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:07.190846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:07.301657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca63b8-6558-4850-b7aa-0d17f764debf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:07.343433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:07.344323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:07.344650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:07.345002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:07.346032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:07.346313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:07.348376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde172f9-cbe6-4f6d-b538-246354fcd003 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:07.355671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 667/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 16:29:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:07.362043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:07.475468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a86f7-315b-4ce5-bf7a-35bee502c1ca None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:07.511965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:07.512583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:07.512860 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:07.513132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] PATH_INFO: `/v3/services` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:07.513806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:07.514114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:07.515495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda549-1f78-4032-91c9-ab3ff3c04398 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:07.520171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 625/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 16:29:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:09.554906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0f452-2b1a-42a2-9990-88a61a4ddaef None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:09.555992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0f452-2b1a-42a2-9990-88a61a4ddaef None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:09.555992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0f452-2b1a-42a2-9990-88a61a4ddaef None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:09.557775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 668/1293] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:29:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:29:09.565840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e986bb-ae7d-4152-b962-372a594fcdca None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:09.566353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e986bb-ae7d-4152-b962-372a594fcdca None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:09.566353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e986bb-ae7d-4152-b962-372a594fcdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:09.587060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a8e986bb-ae7d-4152-b962-372a594fcdca None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:29:09.588395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a8e986bb-ae7d-4152-b962-372a594fcdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:29:09.620323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:09.660843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 626/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:29:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:29:09.683912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0f452-2b1a-42a2-9990-88a61a4ddaef None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:09.718377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:09.793667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0f452-2b1a-42a2-9990-88a61a4ddaef None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:09.831405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0525e-4ee8-4c8d-8750-5838e0cb9422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:09.831979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0525e-4ee8-4c8d-8750-5838e0cb9422 None admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:09.832254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0525e-4ee8-4c8d-8750-5838e0cb9422 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:09.832525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0525e-4ee8-4c8d-8750-5838e0cb9422 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:09.833221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0525e-4ee8-4c8d-8750-5838e0cb9422 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:09.834598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0525e-4ee8-4c8d-8750-5838e0cb9422 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:09.852019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 669/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Aug 1 16:29:09 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:29:11.942467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680a66-a433-4c78-bd8d-c86a5d4511f1 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:11.942981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680a66-a433-4c78-bd8d-c86a5d4511f1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:11.942981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680a66-a433-4c78-bd8d-c86a5d4511f1 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:11.944349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 627/1296] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:29:11.950418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:11.950820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:11.950961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:11.971993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:29:11.973243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:29:12.054509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 670/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:29:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:29:12.077965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680a66-a433-4c78-bd8d-c86a5d4511f1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:12.185386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680a66-a433-4c78-bd8d-c86a5d4511f1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:12.220969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:12.221531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:12.221804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:12.222077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:12.222707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:12.224117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:12.234042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:12.235321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:12.236473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 628/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:29:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:12.241870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:12.260524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:12.347791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4461ef-3e37-4c59-a5b4-37dd6edafb10 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:12.383486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:12.384196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:12.384467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:12.384732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:12.385401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:12.385694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:12.387245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:12.392741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 671/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 16:29:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:12.399609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:12.433144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:12.509309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c8641-108d-49b1-9069-fa9c1f42a50a None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:12.545959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73917393-a915-4362-80fa-489284cd232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:12.546597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73917393-a915-4362-80fa-489284cd232f None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:12.546873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73917393-a915-4362-80fa-489284cd232f None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:12.547134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73917393-a915-4362-80fa-489284cd232f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:12.547905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73917393-a915-4362-80fa-489284cd232f None admin] RBAC: Adding query filter params (interface=public, service_id=a943594c9e07484e9ad8b18f270c8395, region_id=RegionOne) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:12.548205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73917393-a915-4362-80fa-489284cd232f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:12.549584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73917393-a915-4362-80fa-489284cd232f None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:12.555093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 629/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Aug 1 16:29:12 2022] GET /identity/v3/endpoints?service_id=a943594c9e07484e9ad8b18f270c8395&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:12.561521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:12.646179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:12.665452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5d55-3379-45e2-8d8f-b0f22dafd0f0 None admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:12.704538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17473d7d-3819-4556-8051-684f366f62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:12.705186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17473d7d-3819-4556-8051-684f366f62a9 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:12.705462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17473d7d-3819-4556-8051-684f366f62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:12.705732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17473d7d-3819-4556-8051-684f366f62a9 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:12.706394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17473d7d-3819-4556-8051-684f366f62a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:12.707863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17473d7d-3819-4556-8051-684f366f62a9 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:12.716246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 672/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Aug 1 16:29:12 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:29:14.723864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556197f-15c6-43b4-852b-061312fde333 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:14.724505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556197f-15c6-43b4-852b-061312fde333 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:14.724505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556197f-15c6-43b4-852b-061312fde333 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:14.725879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 630/1302] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:29:14.731102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:14.731541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:14.731641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:14.750536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:29:14.751714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:29:14.821922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 673/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:29:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:29:14.844654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556197f-15c6-43b4-852b-061312fde333 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:14.968129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:14.976207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556197f-15c6-43b4-852b-061312fde333 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:15.012484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:15.013127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:15.013403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:15.013672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:15.014303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:15.015690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:29:15.024915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:29:15.026733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application with Lock( Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: ERROR keystone.server.flask.application Aug 01 16:29:15.028361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 631/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Aug 1 16:29:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:29:15.032692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:15.116201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:15.141055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0c2db-53fc-4de4-8b48-6e41b3589450 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:15.177139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:15.177763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:15.178039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:15.178306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] PATH_INFO: `/v3/services` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:15.178986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:29:15.179294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:15.180775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed124e0b-7a3f-45e7-9973-d45a41638957 None admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:15.186460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 674/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Aug 1 16:29:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:29:15.192630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:29:15.307727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ae2bb-fe39-4e23-a5a7-dcc191bf8fb6 None admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:29:15.347233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0bd98-a65e-4b39-8edb-118641b2e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:29:15.347921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0bd98-a65e-4b39-8edb-118641b2e1c9 None admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:29:15.348196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0bd98-a65e-4b39-8edb-118641b2e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:29:15.348464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0bd98-a65e-4b39-8edb-118641b2e1c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:29:15.349194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0bd98-a65e-4b39-8edb-118641b2e1c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:29:15.350756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0bd98-a65e-4b39-8edb-118641b2e1c9 None admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:29:15.378098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 632/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:29:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:29:25.183365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=30 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:29:25.367931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:21.094119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d26b-dcee-4dec-ad3e-5ce01cd1a9ae None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:21.094613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d26b-dcee-4dec-ad3e-5ce01cd1a9ae None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:21.094694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d26b-dcee-4dec-ad3e-5ce01cd1a9ae None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:21.096091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 675/1307] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:21.102911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37dfbc-cb97-4bfc-b126-ab998ced052c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:21.103365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37dfbc-cb97-4bfc-b126-ab998ced052c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:21.103633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37dfbc-cb97-4bfc-b126-ab998ced052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:21.200376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-db37dfbc-cb97-4bfc-b126-ab998ced052c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:21.201610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-db37dfbc-cb97-4bfc-b126-ab998ced052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:21.318583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 633/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:21.373334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143879-8e14-46a8-822e-9e102c430a2d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:21.373610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143879-8e14-46a8-822e-9e102c430a2d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:21.374213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143879-8e14-46a8-822e-9e102c430a2d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:21.375592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 676/1309] 10.222.0.45 () {54 vars in 965 bytes} [Mon Aug 1 16:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:21.381939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d043832-448a-4aac-a76b-5f753e0972ea None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:21.382341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d043832-448a-4aac-a76b-5f753e0972ea None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:21.383108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d043832-448a-4aac-a76b-5f753e0972ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:21.489664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4d043832-448a-4aac-a76b-5f753e0972ea None None] MFA Rules not processed for user `d85bf2b530ce423d8f1c010b5044dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:21.491208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4d043832-448a-4aac-a76b-5f753e0972ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:21.516081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:21.592207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 634/1310] 10.222.0.45 () {58 vars in 1062 bytes} [Mon Aug 1 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:21.594041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143879-8e14-46a8-822e-9e102c430a2d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:21.596296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-37143879-8e14-46a8-822e-9e102c430a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:21.767505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143879-8e14-46a8-822e-9e102c430a2d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:21.856573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:21.857198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:21.857440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:21.857734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:21.859392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:21.921610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:21.984403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:21.986408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd899d42-4114-4fde-91d9-5ae29dd53fc6 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:22.007397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 677/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:24.344845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca2d1-7366-4b34-85e1-883fbdc4495b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:24.345424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca2d1-7366-4b34-85e1-883fbdc4495b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:24.345424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca2d1-7366-4b34-85e1-883fbdc4495b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:24.346757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 635/1312] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:24.353751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc3214-a2d7-4d8d-850c-28284482224f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:24.354335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc3214-a2d7-4d8d-850c-28284482224f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:24.354524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc3214-a2d7-4d8d-850c-28284482224f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:24.448817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-02cc3214-a2d7-4d8d-850c-28284482224f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:24.450038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-02cc3214-a2d7-4d8d-850c-28284482224f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:24.572125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 678/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:24.606287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eca2d1-7366-4b34-85e1-883fbdc4495b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:24.607855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-34eca2d1-7366-4b34-85e1-883fbdc4495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:24.727083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eca2d1-7366-4b34-85e1-883fbdc4495b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:24.804281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:24.807748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:24.808480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:24.808755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:24.809027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:24.811436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:24.936883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:24.938354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b9135-a820-4bd9-8e9e-96d0bc42af23 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:24.958506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 636/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:27.386298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74fd15-d342-4041-a070-3a53fc69ef7f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:27.386937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74fd15-d342-4041-a070-3a53fc69ef7f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:27.386937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74fd15-d342-4041-a070-3a53fc69ef7f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:27.388284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 679/1315] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:27.394157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2901d8-9e69-4c16-bc76-8fa72211de00 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:27.394637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2901d8-9e69-4c16-bc76-8fa72211de00 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:27.394757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2901d8-9e69-4c16-bc76-8fa72211de00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:27.468571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7f2901d8-9e69-4c16-bc76-8fa72211de00 None None] MFA Rules not processed for user `7803329d19ae4e269fa7a06c4a9d2eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:27.470426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7f2901d8-9e69-4c16-bc76-8fa72211de00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:27.532433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:27.582118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 637/1316] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:27.647912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcaefef-3c40-4a41-8056-7434356c45ad None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:27.648139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcaefef-3c40-4a41-8056-7434356c45ad None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:27.648492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcaefef-3c40-4a41-8056-7434356c45ad None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:27.649865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 680/1317] 10.222.0.45 () {54 vars in 965 bytes} [Mon Aug 1 16:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:27.655923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881825c7-5600-4af9-8119-1159f3cb7227 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:27.656210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881825c7-5600-4af9-8119-1159f3cb7227 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:27.656481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881825c7-5600-4af9-8119-1159f3cb7227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:27.741279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-881825c7-5600-4af9-8119-1159f3cb7227 None None] MFA Rules not processed for user `d85bf2b530ce423d8f1c010b5044dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:27.742866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-881825c7-5600-4af9-8119-1159f3cb7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:27.832466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 638/1318] 10.222.0.45 () {58 vars in 1062 bytes} [Mon Aug 1 16:30:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:27.843173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcaefef-3c40-4a41-8056-7434356c45ad None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:27.845771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1bcaefef-3c40-4a41-8056-7434356c45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:27.862484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:27.966745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcaefef-3c40-4a41-8056-7434356c45ad None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:28.045423 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:28.046055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:28.046313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:28.046584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:28.048271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:28.115972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:28.165586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:28.167030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8e587-ded8-4dbc-8097-5bf6a747787f service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:28.186307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 681/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:31.246127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68558c-0977-404d-a2ff-d048ef2b1586 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:31.246768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68558c-0977-404d-a2ff-d048ef2b1586 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:31.246768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68558c-0977-404d-a2ff-d048ef2b1586 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:31.248599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 639/1320] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:31.254097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e763df-b83e-49a1-a4be-119d60d7304d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:31.254617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e763df-b83e-49a1-a4be-119d60d7304d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:31.254719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e763df-b83e-49a1-a4be-119d60d7304d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:31.333051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-18e763df-b83e-49a1-a4be-119d60d7304d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:31.334282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-18e763df-b83e-49a1-a4be-119d60d7304d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:31.445761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 682/1321] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:31.484711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68558c-0977-404d-a2ff-d048ef2b1586 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:31.486620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1b68558c-0977-404d-a2ff-d048ef2b1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:31.545013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:31.605115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68558c-0977-404d-a2ff-d048ef2b1586 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:31.685137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:31.685758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:31.686031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:31.686729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:31.687965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:31.792870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:31.807155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:31.808579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36494e6-e517-4424-bdd0-5ba03a8a9462 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:31.828182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 640/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:34.692682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aca7e7-e683-4703-b5e3-1ad3732a156d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:34.693301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aca7e7-e683-4703-b5e3-1ad3732a156d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:34.693301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aca7e7-e683-4703-b5e3-1ad3732a156d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:34.694633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 683/1323] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:34.700585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94600b3e-8b87-41dc-8d75-26dc9ad92699 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:34.701116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94600b3e-8b87-41dc-8d75-26dc9ad92699 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:34.701116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94600b3e-8b87-41dc-8d75-26dc9ad92699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:34.783658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-94600b3e-8b87-41dc-8d75-26dc9ad92699 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:34.784858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-94600b3e-8b87-41dc-8d75-26dc9ad92699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:34.893622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 641/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:34.937130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aca7e7-e683-4703-b5e3-1ad3732a156d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:34.939939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b6aca7e7-e683-4703-b5e3-1ad3732a156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:34.967104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:35.080368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aca7e7-e683-4703-b5e3-1ad3732a156d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:35.173335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:35.173970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:35.174244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:35.174517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:35.176268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:35.257646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:35.307277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:35.308784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ac38c-6c7d-4ed9-b2de-39da9a4d91aa service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:35.330570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 684/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:37.498439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb791-c71f-4f46-8a05-30aa3f8b4ea2 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:37.499273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb791-c71f-4f46-8a05-30aa3f8b4ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:37.499273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb791-c71f-4f46-8a05-30aa3f8b4ea2 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:37.500590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 642/1326] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:37.505751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a26c4-985d-482b-a1ff-f842478ae354 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:37.506253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a26c4-985d-482b-a1ff-f842478ae354 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:37.506371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a26c4-985d-482b-a1ff-f842478ae354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:37.579160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-357a26c4-985d-482b-a1ff-f842478ae354 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:37.580305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-357a26c4-985d-482b-a1ff-f842478ae354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:37.687774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 685/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:37.721263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb791-c71f-4f46-8a05-30aa3f8b4ea2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:37.723001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-445cb791-c71f-4f46-8a05-30aa3f8b4ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:37.788591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:37.834173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb791-c71f-4f46-8a05-30aa3f8b4ea2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:37.923953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:37.924592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:37.924909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:37.925126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:37.926845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:38.058546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:38.061223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:38.063049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c566dc-6f69-413c-9719-88e0001994f8 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:38.085328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 643/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:40.159795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d87b1-9109-4712-8a23-24832920addc None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:40.160393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d87b1-9109-4712-8a23-24832920addc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:40.160503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d87b1-9109-4712-8a23-24832920addc None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:40.162021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 686/1329] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:40.167604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d897a0-f8c9-4813-ae7b-aad85435faad None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:40.168115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d897a0-f8c9-4813-ae7b-aad85435faad None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:40.168204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d897a0-f8c9-4813-ae7b-aad85435faad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:40.239008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-76d897a0-f8c9-4813-ae7b-aad85435faad None None] MFA Rules not processed for user `7803329d19ae4e269fa7a06c4a9d2eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:40.240234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-76d897a0-f8c9-4813-ae7b-aad85435faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:40.347132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 644/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:40.387156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d87b1-9109-4712-8a23-24832920addc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:40.388861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-511d87b1-9109-4712-8a23-24832920addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:40.433058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:40.504712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d87b1-9109-4712-8a23-24832920addc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:40.582975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:40.583509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:40.583804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:40.584086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:40.585730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:40.677584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:40.712752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:40.714223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfed5de-a716-4abd-84a1-813276600a1c service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:40.732588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 687/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:43.047047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caff82e-e08c-4943-bf19-bc74474bcd72 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:43.047935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caff82e-e08c-4943-bf19-bc74474bcd72 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:43.047935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caff82e-e08c-4943-bf19-bc74474bcd72 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:43.049968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 645/1332] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:43.058010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc57b26-1e9f-4521-bbe4-45eb95d7ee4b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:43.058534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc57b26-1e9f-4521-bbe4-45eb95d7ee4b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:43.058534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc57b26-1e9f-4521-bbe4-45eb95d7ee4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:43.142015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fcc57b26-1e9f-4521-bbe4-45eb95d7ee4b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:43.143258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fcc57b26-1e9f-4521-bbe4-45eb95d7ee4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:43.257788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 688/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:30:43.296504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caff82e-e08c-4943-bf19-bc74474bcd72 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:43.298102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0caff82e-e08c-4943-bf19-bc74474bcd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:43.381087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:43.422496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caff82e-e08c-4943-bf19-bc74474bcd72 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:43.500130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:43.500750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:43.501022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:43.501288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:43.502923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:43.618882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:43.620277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc1eca-acd3-4d37-ab8b-edce00c3823f service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:43.630429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:43.639224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 646/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:45.715194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0f9c-61a5-472f-9d14-ae31edf90bee None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:45.715755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0f9c-61a5-472f-9d14-ae31edf90bee None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:45.715755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0f9c-61a5-472f-9d14-ae31edf90bee None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:45.717116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 689/1335] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:45.723278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37d09-12ca-4b4b-9f31-a8571a284304 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:45.723755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37d09-12ca-4b4b-9f31-a8571a284304 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:45.723872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37d09-12ca-4b4b-9f31-a8571a284304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:45.801556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-90f37d09-12ca-4b4b-9f31-a8571a284304 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:45.802832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-90f37d09-12ca-4b4b-9f31-a8571a284304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:45.916874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 647/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:45.958196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f0f9c-61a5-472f-9d14-ae31edf90bee None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:45.959838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8e1f0f9c-61a5-472f-9d14-ae31edf90bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:46.005431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:46.079938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f0f9c-61a5-472f-9d14-ae31edf90bee None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:46.175922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:46.176560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:46.176831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:46.177096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:46.178774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:46.265189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:46.294958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:46.297063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a473eec-d22d-413e-a1a4-53f47e609743 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:46.316787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 690/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:48.722498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b64c-bf6c-4b1a-bc52-b0fe8abfdb88 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:48.723188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b64c-bf6c-4b1a-bc52-b0fe8abfdb88 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:48.723188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b64c-bf6c-4b1a-bc52-b0fe8abfdb88 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:48.724457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 648/1338] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:48.729978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b4243-b649-4b01-8569-84dfec3d1580 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:48.730462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b4243-b649-4b01-8569-84dfec3d1580 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:48.730462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b4243-b649-4b01-8569-84dfec3d1580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:48.800432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-991b4243-b649-4b01-8569-84dfec3d1580 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:48.801545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-991b4243-b649-4b01-8569-84dfec3d1580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:48.906064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 691/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:30:48.942984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b64c-bf6c-4b1a-bc52-b0fe8abfdb88 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:48.945364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-68c7b64c-bf6c-4b1a-bc52-b0fe8abfdb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:49.024419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:49.054192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b64c-bf6c-4b1a-bc52-b0fe8abfdb88 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:49.138640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:49.139324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:49.139602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:49.139942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:49.141625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:49.259337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:49.260812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de250-0447-4eb1-9fe1-d7e5b52c81b7 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:49.280023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 649/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:53.330855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c1f7a-68cc-46f9-ab8a-4ea2ec29955e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:53.331480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c1f7a-68cc-46f9-ab8a-4ea2ec29955e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:53.331480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c1f7a-68cc-46f9-ab8a-4ea2ec29955e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:53.332817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 692/1341] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:53.338192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d229-6a9b-49d1-ad0f-77f40088ac3d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:53.338632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d229-6a9b-49d1-ad0f-77f40088ac3d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:53.338748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d229-6a9b-49d1-ad0f-77f40088ac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:53.344417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:53.410128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-47e6d229-6a9b-49d1-ad0f-77f40088ac3d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:53.411352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-47e6d229-6a9b-49d1-ad0f-77f40088ac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:53.512843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 650/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:53.550718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c1f7a-68cc-46f9-ab8a-4ea2ec29955e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:53.552377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-717c1f7a-68cc-46f9-ab8a-4ea2ec29955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:53.605084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:53.662736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c1f7a-68cc-46f9-ab8a-4ea2ec29955e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:53.745314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:53.745943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:53.746216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:53.746484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:53.748182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:53.849083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:53.869634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:53.871095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a4bcd-e279-41d1-b38c-a3a7e208e556 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:53.889834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 693/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:30:56.681883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d1e91-7541-4567-972c-1a2db9a9b575 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:56.682378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d1e91-7541-4567-972c-1a2db9a9b575 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:56.682468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d1e91-7541-4567-972c-1a2db9a9b575 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:56.683857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 651/1344] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:30:56.689389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c85277-e11c-45c6-a396-c5264efa2a46 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:56.689861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c85277-e11c-45c6-a396-c5264efa2a46 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:56.689973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c85277-e11c-45c6-a396-c5264efa2a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:56.759810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b0c85277-e11c-45c6-a396-c5264efa2a46 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:30:56.761066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b0c85277-e11c-45c6-a396-c5264efa2a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:56.862416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 694/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:30:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:30:56.905607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d1e91-7541-4567-972c-1a2db9a9b575 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:30:56.907155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-004d1e91-7541-4567-972c-1a2db9a9b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:56.999330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:30:57.017804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d1e91-7541-4567-972c-1a2db9a9b575 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:30:57.093177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:30:57.093806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:30:57.094084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:30:57.094373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:30:57.096033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:30:57.229442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:30:57.230988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c77bd-2111-442f-85c4-f01997cb2b01 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:30:57.256361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 652/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:30:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:01.343133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92a2e0-f28f-4017-91ae-04408076d4d3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:01.343711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92a2e0-f28f-4017-91ae-04408076d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:01.343711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92a2e0-f28f-4017-91ae-04408076d4d3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:01.345262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 695/1347] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:01.351082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f43053-283a-41ac-90ed-ef608d0816d2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:01.351580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f43053-283a-41ac-90ed-ef608d0816d2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:01.351702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f43053-283a-41ac-90ed-ef608d0816d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:01.369240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:01.438699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d6f43053-283a-41ac-90ed-ef608d0816d2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:01.439951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6f43053-283a-41ac-90ed-ef608d0816d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:01.560569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 653/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:01.599513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92a2e0-f28f-4017-91ae-04408076d4d3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:01.601140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2f92a2e0-f28f-4017-91ae-04408076d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:01.664641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:01.714992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92a2e0-f28f-4017-91ae-04408076d4d3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:01.796403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:01.797043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:01.797319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:01.797593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:01.799275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:01.926702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:01.930950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:01.932936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca937b42-2fb4-4fdf-99b7-58678b0ea235 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:01.952201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 696/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:07.059554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388c099-f8d3-4190-b836-93dfed74b981 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:07.059979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388c099-f8d3-4190-b836-93dfed74b981 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:07.060260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388c099-f8d3-4190-b836-93dfed74b981 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:07.062150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 654/1350] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:07.067958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393fe1b-0c95-4ddf-9a47-c69602d7cb5f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:07.068831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393fe1b-0c95-4ddf-9a47-c69602d7cb5f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:07.070592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393fe1b-0c95-4ddf-9a47-c69602d7cb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:07.177426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3393fe1b-0c95-4ddf-9a47-c69602d7cb5f None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:07.178846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3393fe1b-0c95-4ddf-9a47-c69602d7cb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:07.324961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 697/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:31:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:11.558449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=59 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:11.657426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1015-8a1b-4fba-b15e-a4677b885bbc None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:11.657822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1015-8a1b-4fba-b15e-a4677b885bbc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:11.658052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1015-8a1b-4fba-b15e-a4677b885bbc None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:11.659584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 655/1352] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:11.665118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d6f2-16b7-4831-8e2e-eae386b4931a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:11.666254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d6f2-16b7-4831-8e2e-eae386b4931a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:11.666893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d6f2-16b7-4831-8e2e-eae386b4931a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:11.738614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:11.765308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4a93d6f2-16b7-4831-8e2e-eae386b4931a None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:11.767413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4a93d6f2-16b7-4831-8e2e-eae386b4931a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:11.902701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 698/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:31:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:13.194176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce5479-21e3-47f4-816b-6aca2065179a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:13.194771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce5479-21e3-47f4-816b-6aca2065179a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:13.194771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce5479-21e3-47f4-816b-6aca2065179a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:13.196126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 656/1354] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:13.201911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b99f-8a2d-4f93-b928-6bc32e110223 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:13.202416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b99f-8a2d-4f93-b928-6bc32e110223 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:13.202644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b99f-8a2d-4f93-b928-6bc32e110223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:13.289090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4525b99f-8a2d-4f93-b928-6bc32e110223 None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:13.290432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4525b99f-8a2d-4f93-b928-6bc32e110223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:13.333981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:13.413496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 699/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:31:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:16.884357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dd463-d903-4b7a-8045-e36456e19f48 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:16.884939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dd463-d903-4b7a-8045-e36456e19f48 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:16.884939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dd463-d903-4b7a-8045-e36456e19f48 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:16.886291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 657/1356] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:16.891725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a912e-3802-486c-ab28-ffca52774be5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:16.892328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a912e-3802-486c-ab28-ffca52774be5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:16.892328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a912e-3802-486c-ab28-ffca52774be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:16.988921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a56a912e-3802-486c-ab28-ffca52774be5 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:16.991837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a56a912e-3802-486c-ab28-ffca52774be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:17.111983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 700/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:17.169738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836422-6c18-4fe7-93bb-d6377797ccb9 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:17.170704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836422-6c18-4fe7-93bb-d6377797ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:17.171513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f836422-6c18-4fe7-93bb-d6377797ccb9 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:17.173491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 658/1358] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 16:31:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:17.183571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebce317-606e-46f1-a02f-00c4f74b2046 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:17.184724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebce317-606e-46f1-a02f-00c4f74b2046 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:17.185751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebce317-606e-46f1-a02f-00c4f74b2046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:17.206675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:17.281395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7ebce317-606e-46f1-a02f-00c4f74b2046 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:17.282747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7ebce317-606e-46f1-a02f-00c4f74b2046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:17.422646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 701/1359] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 16:31:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:17.430741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836422-6c18-4fe7-93bb-d6377797ccb9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:17.432509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4f836422-6c18-4fe7-93bb-d6377797ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:17.602619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f836422-6c18-4fe7-93bb-d6377797ccb9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:17.689386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:17.734927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:17.735700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:17.735929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:17.736571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:17.739088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:17.846692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:17.848665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3551-c92d-4ea7-9c87-040a69cf05db service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:17.870451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 659/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:20.129336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d9e78-099f-4bfe-a6c5-71b4f31569a4 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:20.129785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d9e78-099f-4bfe-a6c5-71b4f31569a4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:20.130004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d9e78-099f-4bfe-a6c5-71b4f31569a4 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:20.132037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 702/1361] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:20.137972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34264-5a66-42fb-8cac-dd8468cf737f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:20.138484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34264-5a66-42fb-8cac-dd8468cf737f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:20.138746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34264-5a66-42fb-8cac-dd8468cf737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:20.219032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d8e34264-5a66-42fb-8cac-dd8468cf737f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:20.220194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d8e34264-5a66-42fb-8cac-dd8468cf737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:20.263198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:20.332990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 660/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:20.380441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d9e78-099f-4bfe-a6c5-71b4f31569a4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:20.382100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bb0d9e78-099f-4bfe-a6c5-71b4f31569a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:20.458862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:20.561171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d9e78-099f-4bfe-a6c5-71b4f31569a4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:20.708271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:20.708902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:20.709183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:20.709436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:20.711050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:20.737937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:20.808838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:20.810479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1e596-84df-4672-87cf-1848b6f72e91 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:20.830451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 703/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:23.256932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb4e6d-9e6a-4ead-ab68-7233009625f3 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:23.257479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb4e6d-9e6a-4ead-ab68-7233009625f3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:23.257479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb4e6d-9e6a-4ead-ab68-7233009625f3 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:23.258927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 661/1364] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:23.264981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08f48f-26d6-4a2e-b175-58e552433fdd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:23.265946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08f48f-26d6-4a2e-b175-58e552433fdd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:23.266219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08f48f-26d6-4a2e-b175-58e552433fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:23.349946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ae08f48f-26d6-4a2e-b175-58e552433fdd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:23.351737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ae08f48f-26d6-4a2e-b175-58e552433fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:23.456414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:23.465861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 704/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:23.505564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434936-a6aa-424a-8a96-afc8b07a5398 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:23.505864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434936-a6aa-424a-8a96-afc8b07a5398 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:23.506108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434936-a6aa-424a-8a96-afc8b07a5398 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:23.507654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 662/1366] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:23.514508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6d892-b64b-432d-97b7-23d9a8397962 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:23.514869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6d892-b64b-432d-97b7-23d9a8397962 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:23.515215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6d892-b64b-432d-97b7-23d9a8397962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:23.604963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fce6d892-b64b-432d-97b7-23d9a8397962 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:23.606171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fce6d892-b64b-432d-97b7-23d9a8397962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:23.739812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 705/1367] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 16:31:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:23.749350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434936-a6aa-424a-8a96-afc8b07a5398 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:23.750964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-56434936-a6aa-424a-8a96-afc8b07a5398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:23.918039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434936-a6aa-424a-8a96-afc8b07a5398 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:23.941013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:24.044892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:24.045520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:24.045800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:24.046073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:24.047709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:24.144001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:24.145636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed1749-56d6-46af-98fe-ab0d23d223a8 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:24.166771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 663/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:25.140086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4388bb9-53d7-4334-b1a9-afc85123f5bf None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:25.140633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4388bb9-53d7-4334-b1a9-afc85123f5bf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:25.140716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4388bb9-53d7-4334-b1a9-afc85123f5bf None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:25.142366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 706/1369] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:25.148113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389abb1-9d7b-4d7d-8bfb-6de617717a56 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:25.148570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389abb1-9d7b-4d7d-8bfb-6de617717a56 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:25.148650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389abb1-9d7b-4d7d-8bfb-6de617717a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:25.186511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:25.228320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8389abb1-9d7b-4d7d-8bfb-6de617717a56 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:25.230061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8389abb1-9d7b-4d7d-8bfb-6de617717a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:25.348206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 664/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:25.662950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0be86-9a65-4681-b071-ce40b32aec35 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:25.663303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0be86-9a65-4681-b071-ce40b32aec35 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:25.663522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0be86-9a65-4681-b071-ce40b32aec35 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:25.665595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 707/1371] 10.222.0.45 () {54 vars in 964 bytes} [Mon Aug 1 16:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:25.676744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ebac-d605-4dc8-8b4c-1168e56952f2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:25.677057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ebac-d605-4dc8-8b4c-1168e56952f2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:25.677298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ebac-d605-4dc8-8b4c-1168e56952f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:25.768340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-13c6ebac-d605-4dc8-8b4c-1168e56952f2 None None] MFA Rules not processed for user `698e47aac57c48adb8836cec7226b039`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:25.769838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-13c6ebac-d605-4dc8-8b4c-1168e56952f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:25.824757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:25.859531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 665/1372] 10.222.0.45 () {58 vars in 1061 bytes} [Mon Aug 1 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:25.867446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0be86-9a65-4681-b071-ce40b32aec35 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:25.869275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ada0be86-9a65-4681-b071-ce40b32aec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:25.930670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:25.996380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0be86-9a65-4681-b071-ce40b32aec35 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:26.083454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:26.084236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:26.084515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:26.084780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:26.086415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:26.200994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:26.216983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:26.218876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d5522-fa6e-4ee0-8e00-6d0dd26d5f84 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:26.240681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 708/1373] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:26.259195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013108-6e16-46e5-8af2-21eb4e50ad12 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:26.259938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013108-6e16-46e5-8af2-21eb4e50ad12 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:26.259938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013108-6e16-46e5-8af2-21eb4e50ad12 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:26.264563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 666/1374] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:26.267638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6f513-9e35-4302-9693-0f1e637319c0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:26.267885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6f513-9e35-4302-9693-0f1e637319c0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:26.268161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6f513-9e35-4302-9693-0f1e637319c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:26.343376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-88e6f513-9e35-4302-9693-0f1e637319c0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:26.344549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88e6f513-9e35-4302-9693-0f1e637319c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:26.446927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 709/1375] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:26.495439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013108-6e16-46e5-8af2-21eb4e50ad12 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:26.497016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c013108-6e16-46e5-8af2-21eb4e50ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:26.664058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013108-6e16-46e5-8af2-21eb4e50ad12 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:26.717745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:26.781265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:26.782010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:26.782319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:26.782653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:26.784396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:26.877506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:26.879239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d286a53-1edd-4409-b0fc-33e275117fd2 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:26.898775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 667/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:28.966939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d06445-ecda-457c-bb95-7d5b228b52e9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:28.967502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d06445-ecda-457c-bb95-7d5b228b52e9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:28.967502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d06445-ecda-457c-bb95-7d5b228b52e9 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:28.968809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 710/1377] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:28.974711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bcb29-fe72-49ae-bcdd-4c4808d5a16b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:28.975192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bcb29-fe72-49ae-bcdd-4c4808d5a16b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:28.975291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bcb29-fe72-49ae-bcdd-4c4808d5a16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:29.045930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0a2bcb29-fe72-49ae-bcdd-4c4808d5a16b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:29.047121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0a2bcb29-fe72-49ae-bcdd-4c4808d5a16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:29.054420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:29.149036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 668/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:29.199753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d06445-ecda-457c-bb95-7d5b228b52e9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:29.201323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-17d06445-ecda-457c-bb95-7d5b228b52e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:29.272318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:29.384123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d06445-ecda-457c-bb95-7d5b228b52e9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:29.522891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:29.523597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:29.523899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:29.524218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:29.525881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:29.546662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:29.625807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:29.627426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81101e-7252-4b50-9a35-156b0d47ca15 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:29.646714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 711/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:31.816333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f48677-5f0a-456c-94ce-fbdac111caa1 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:31.816888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f48677-5f0a-456c-94ce-fbdac111caa1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:31.816888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f48677-5f0a-456c-94ce-fbdac111caa1 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:31.818441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 669/1380] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:31.823751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba0b05-c11b-41ee-ad1e-f29414d816dd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:31.824269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba0b05-c11b-41ee-ad1e-f29414d816dd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:31.824355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba0b05-c11b-41ee-ad1e-f29414d816dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:31.898951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4fba0b05-c11b-41ee-ad1e-f29414d816dd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:31.900127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4fba0b05-c11b-41ee-ad1e-f29414d816dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:32.007863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:32.023313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 712/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:32.061232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f48677-5f0a-456c-94ce-fbdac111caa1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:32.062772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-14f48677-5f0a-456c-94ce-fbdac111caa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:32.222483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:32.234814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f48677-5f0a-456c-94ce-fbdac111caa1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:32.378849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:32.379597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:32.380105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:32.380460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:32.382544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:32.481863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:32.483397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184cc29-a90d-4706-a980-658ec83733bd service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:32.496255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:32.506159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 670/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:34.760853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb6380-b1f7-45fa-a210-4545bc1fb2b8 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:34.761467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb6380-b1f7-45fa-a210-4545bc1fb2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:34.761467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb6380-b1f7-45fa-a210-4545bc1fb2b8 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:34.762812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 713/1383] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:34.768557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8848d-95f7-4e66-856c-61f021b64b86 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:34.769062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8848d-95f7-4e66-856c-61f021b64b86 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:34.769159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8848d-95f7-4e66-856c-61f021b64b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:34.854637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-03a8848d-95f7-4e66-856c-61f021b64b86 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:34.856734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-03a8848d-95f7-4e66-856c-61f021b64b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:34.972674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 671/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:35.017892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb6380-b1f7-45fa-a210-4545bc1fb2b8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:35.019481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-afeb6380-b1f7-45fa-a210-4545bc1fb2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:35.181690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb6380-b1f7-45fa-a210-4545bc1fb2b8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:35.253703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:35.308693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:35.309470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:35.309813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:35.310454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:35.312890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:35.420412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:35.422096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f42475-ac01-4e77-ab3e-0568c55eae86 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:35.441753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 714/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:37.570898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1b910-4cfb-45a0-9b9c-ae0951596693 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:37.571568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1b910-4cfb-45a0-9b9c-ae0951596693 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:37.571568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1b910-4cfb-45a0-9b9c-ae0951596693 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:37.572945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 672/1386] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:37.578580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe9f02-19c8-41df-8126-ebeaae70add9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:37.579127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe9f02-19c8-41df-8126-ebeaae70add9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:37.579307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe9f02-19c8-41df-8126-ebeaae70add9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:37.651668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3dbe9f02-19c8-41df-8126-ebeaae70add9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:37.652938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3dbe9f02-19c8-41df-8126-ebeaae70add9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:37.692647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:37.783324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 715/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:37.833259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1b910-4cfb-45a0-9b9c-ae0951596693 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:37.834863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b1f1b910-4cfb-45a0-9b9c-ae0951596693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:37.926433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:38.007327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1b910-4cfb-45a0-9b9c-ae0951596693 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:38.128299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:38.128949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:38.129417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:38.129717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:38.131371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:38.172697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:38.229657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:38.231348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95739f49-44e5-4111-ae68-6c575684f49d service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:38.251060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 673/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:40.364056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fe6e1-9787-4121-af22-a37cf512451d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:40.364677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fe6e1-9787-4121-af22-a37cf512451d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:40.364677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fe6e1-9787-4121-af22-a37cf512451d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:40.365986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 716/1389] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:40.371514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9574c-b713-4cb7-9205-9c5f2e536642 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:40.372023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9574c-b713-4cb7-9205-9c5f2e536642 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:40.372134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9574c-b713-4cb7-9205-9c5f2e536642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:40.439440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4ea9574c-b713-4cb7-9205-9c5f2e536642 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:40.440711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4ea9574c-b713-4cb7-9205-9c5f2e536642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:40.542560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 674/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:40.590799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fe6e1-9787-4121-af22-a37cf512451d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:40.592366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-196fe6e1-9787-4121-af22-a37cf512451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:40.759052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fe6e1-9787-4121-af22-a37cf512451d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:40.777203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:40.879836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:40.880469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:40.880745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:40.881015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:40.882649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:40.976691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:40.978299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bcf49-c721-4d29-963d-165b87d89445 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:40.998309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 717/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:41.636083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2006171-410d-4fbe-a853-07f9ae3e9c2a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:41.636687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2006171-410d-4fbe-a853-07f9ae3e9c2a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:41.636687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2006171-410d-4fbe-a853-07f9ae3e9c2a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:41.638040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 675/1392] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:41.643431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071f115-efe2-413e-ac17-b138dac433c7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:41.643911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071f115-efe2-413e-ac17-b138dac433c7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:41.644018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071f115-efe2-413e-ac17-b138dac433c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:41.671718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:41.715182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9071f115-efe2-413e-ac17-b138dac433c7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:41.716447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9071f115-efe2-413e-ac17-b138dac433c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:41.821393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 718/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:41.877435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166cc14-4e43-4b38-9157-7fb76fcdba25 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:41.877714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166cc14-4e43-4b38-9157-7fb76fcdba25 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:41.877986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166cc14-4e43-4b38-9157-7fb76fcdba25 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:41.879354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 676/1394] 10.222.0.45 () {54 vars in 966 bytes} [Mon Aug 1 16:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:41.884791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef57b4-8748-4113-93c0-c850f10e6f6b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:41.885088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef57b4-8748-4113-93c0-c850f10e6f6b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:41.885480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef57b4-8748-4113-93c0-c850f10e6f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:41.967681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4cef57b4-8748-4113-93c0-c850f10e6f6b None None] MFA Rules not processed for user `288250b6b648409aa179b3213321b869`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:41.968848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4cef57b4-8748-4113-93c0-c850f10e6f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:42.010982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:42.049839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 719/1395] 10.222.0.45 () {58 vars in 1063 bytes} [Mon Aug 1 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:42.057792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166cc14-4e43-4b38-9157-7fb76fcdba25 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:42.059338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5166cc14-4e43-4b38-9157-7fb76fcdba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:42.073624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:42.175118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166cc14-4e43-4b38-9157-7fb76fcdba25 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:42.253308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:42.253983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:42.254253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:42.254535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:42.256319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:42.325369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:42.373098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:42.374569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f841a73-2897-451a-9ed9-4c1b2135431b service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:42.394072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 677/1396] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:43.062258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e546b-1b80-41f9-a1dd-37b211626781 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.062868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e546b-1b80-41f9-a1dd-37b211626781 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.062868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e546b-1b80-41f9-a1dd-37b211626781 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.064239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 720/1397] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:43.070325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.070894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.070894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.145255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:43.146422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:43.157422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7610a74-f9fb-43ab-a71f-40602dfee3fd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.157724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7610a74-f9fb-43ab-a71f-40602dfee3fd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.158067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7610a74-f9fb-43ab-a71f-40602dfee3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.227291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a7610a74-f9fb-43ab-a71f-40602dfee3fd None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:43.228974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a7610a74-f9fb-43ab-a71f-40602dfee3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:43.253939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 678/1398] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:43.301928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:43.303275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:43.326570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 721/1399] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:43.333325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:43.377427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff165-ff4e-4325-a9b6-decc02189c89 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.377838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff165-ff4e-4325-a9b6-decc02189c89 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.378113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff165-ff4e-4325-a9b6-decc02189c89 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.379633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 722/1400] 10.222.0.45 () {54 vars in 943 bytes} [Mon Aug 1 16:31:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:31:43.385942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.386218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.386510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.434980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425d1a2-fd68-41d2-b3b8-156128162149 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:43.458386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:43.466408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] MFA Rules not processed for user `4177b0f6e8204e5285ee4fdaeb2aa4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:43.467632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:43.554052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:43.554691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.554953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.555246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.557010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:43.576398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:43.583171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 723/1401] 10.222.0.45 () {58 vars in 1040 bytes} [Mon Aug 1 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:43.593426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:43.603296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a73fd-5b0f-4035-b27b-d3a0f0e9677c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:43.652250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:43.653877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d4948-41dd-4ecf-8d8d-154892400968 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:43.674331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 679/1402] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:43.721612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:43.722236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:43.722508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:43.722776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:43.724469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:43.727048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:43.830646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:43.832370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c53ce-a857-4b81-8fb8-7d5b798fd8d2 service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:43.842147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 724/1403] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:31:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 16:31:45.740414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72046149-dd44-4211-9560-6c6fc1394b94 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:45.741395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72046149-dd44-4211-9560-6c6fc1394b94 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:45.741724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72046149-dd44-4211-9560-6c6fc1394b94 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:45.743214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 680/1404] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:45.751038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977d65-4d8f-4bf4-962e-bbe14754f690 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:45.751958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977d65-4d8f-4bf4-962e-bbe14754f690 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:45.754235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977d65-4d8f-4bf4-962e-bbe14754f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:45.838144 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4c977d65-4d8f-4bf4-962e-bbe14754f690 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:45.839696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4c977d65-4d8f-4bf4-962e-bbe14754f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:45.975278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:45.986430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 725/1405] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:31:46.043276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72046149-dd44-4211-9560-6c6fc1394b94 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:46.046912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-72046149-dd44-4211-9560-6c6fc1394b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:46.323630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:46.327535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72046149-dd44-4211-9560-6c6fc1394b94 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:46.559982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:46.560626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:46.560904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:46.561180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:46.562991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:46.689671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:46.691317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15bd14-6060-4338-862e-88df31f3172e service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:46.716224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 681/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:48.785837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47ccac-33a2-4b4e-ab20-6d423fc3d47c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:48.786412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47ccac-33a2-4b4e-ab20-6d423fc3d47c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:48.786412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47ccac-33a2-4b4e-ab20-6d423fc3d47c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:48.787786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 726/1407] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:31:48.793929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894525f-0495-4593-8abd-7804ab33018a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:48.794538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894525f-0495-4593-8abd-7804ab33018a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:48.794538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894525f-0495-4593-8abd-7804ab33018a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:48.799738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:48.878261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f894525f-0495-4593-8abd-7804ab33018a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:31:48.879418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f894525f-0495-4593-8abd-7804ab33018a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:48.992469 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 682/1408] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:31:49.039664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47ccac-33a2-4b4e-ab20-6d423fc3d47c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:31:49.041298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ce47ccac-33a2-4b4e-ab20-6d423fc3d47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:49.211581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47ccac-33a2-4b4e-ab20-6d423fc3d47c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:31:49.287343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:49.344501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:31:49.345160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:31:49.345435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:31:49.345710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:31:49.347559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:31:49.452680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:31:49.454337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7cb4e-4159-4c88-9a08-2eb45c1a0326 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:31:49.474602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 727/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:31:58.988733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=64 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:31:59.472109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=64 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:06.358678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d3c09-bce9-44a8-81e2-dab60e4f0ff6 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:06.359269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d3c09-bce9-44a8-81e2-dab60e4f0ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:06.359269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d3c09-bce9-44a8-81e2-dab60e4f0ff6 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:06.360664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 683/1410] 10.222.0.45 () {54 vars in 918 bytes} [Mon Aug 1 16:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:06.367422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73c14b-1c43-406d-be50-3ed3e958bcfb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:06.368045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73c14b-1c43-406d-be50-3ed3e958bcfb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:06.368045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73c14b-1c43-406d-be50-3ed3e958bcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:06.475658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0d73c14b-1c43-406d-be50-3ed3e958bcfb None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:06.477548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0d73c14b-1c43-406d-be50-3ed3e958bcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:06.624549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 728/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Aug 1 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:06.643896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffda2c-f662-4a4f-8653-8615f2e623e1 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:06.644291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffda2c-f662-4a4f-8653-8615f2e623e1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:06.644664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffda2c-f662-4a4f-8653-8615f2e623e1 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:06.646763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 684/1412] 10.222.0.45 () {54 vars in 918 bytes} [Mon Aug 1 16:32:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:06.652611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee7b2-76e6-4b13-91c7-f62eb4e0c2a7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:06.652894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee7b2-76e6-4b13-91c7-f62eb4e0c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:06.653217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee7b2-76e6-4b13-91c7-f62eb4e0c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:06.754281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:06.757685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-900ee7b2-76e6-4b13-91c7-f62eb4e0c2a7 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:06.759292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-900ee7b2-76e6-4b13-91c7-f62eb4e0c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:06.903392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 729/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Aug 1 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:09.498437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0c283-1b26-4c85-97dd-531bc9cbfdd2 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:09.499012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0c283-1b26-4c85-97dd-531bc9cbfdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:09.499012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0c283-1b26-4c85-97dd-531bc9cbfdd2 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:09.500430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 685/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:09.506203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7554b8-f62b-4c5f-9d92-266f0dbb6ad3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:09.506702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7554b8-f62b-4c5f-9d92-266f0dbb6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:09.506803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7554b8-f62b-4c5f-9d92-266f0dbb6ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:09.580816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2c7554b8-f62b-4c5f-9d92-266f0dbb6ad3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:09.582046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2c7554b8-f62b-4c5f-9d92-266f0dbb6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:09.644500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:09.693436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 730/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:32:09.738068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0c283-1b26-4c85-97dd-531bc9cbfdd2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:09.739630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c5c0c283-1b26-4c85-97dd-531bc9cbfdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:09.905967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0c283-1b26-4c85-97dd-531bc9cbfdd2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:09.984998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:10.031038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:10.031666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:10.031967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:10.032236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:10.033875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:10.130628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:10.133133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6f954-7619-4131-af60-49700930ff87 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:10.153448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 686/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:11.109315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46316669-5110-4be6-b6a5-a572a33c1bd9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:11.109898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46316669-5110-4be6-b6a5-a572a33c1bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:11.109898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46316669-5110-4be6-b6a5-a572a33c1bd9 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:11.111234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 731/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:11.117137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c09536-e792-467b-b036-c6b8f0650b28 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:11.117663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c09536-e792-467b-b036-c6b8f0650b28 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:11.117767 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c09536-e792-467b-b036-c6b8f0650b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:11.209004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c9c09536-e792-467b-b036-c6b8f0650b28 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:11.210250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c9c09536-e792-467b-b036-c6b8f0650b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:11.241791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:11.318329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 687/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:11.474808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46316669-5110-4be6-b6a5-a572a33c1bd9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:11.476251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-46316669-5110-4be6-b6a5-a572a33c1bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:11.640933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46316669-5110-4be6-b6a5-a572a33c1bd9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:11.685024 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:11.772549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9492-8945-4847-8354-333bb195726c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:11.773180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9492-8945-4847-8354-333bb195726c service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:11.773435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9492-8945-4847-8354-333bb195726c service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:11.773716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9492-8945-4847-8354-333bb195726c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:11.775407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-49ca9492-8945-4847-8354-333bb195726c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:11.873800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9492-8945-4847-8354-333bb195726c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:11.875461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9492-8945-4847-8354-333bb195726c service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:11.899290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 732/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:12.803583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d71a5-c892-417e-b8bf-864be1d8d5ef None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:12.804204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d71a5-c892-417e-b8bf-864be1d8d5ef None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:12.804204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d71a5-c892-417e-b8bf-864be1d8d5ef None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:12.805621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 688/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:12.811137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149cd4d-57ba-4760-8490-7e51197b2b5d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:12.811604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149cd4d-57ba-4760-8490-7e51197b2b5d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:12.811707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149cd4d-57ba-4760-8490-7e51197b2b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:12.878493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:12.905095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1149cd4d-57ba-4760-8490-7e51197b2b5d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:12.906310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1149cd4d-57ba-4760-8490-7e51197b2b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:13.014409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 733/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:13.169361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d71a5-c892-417e-b8bf-864be1d8d5ef None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:13.170814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7f3d71a5-c892-417e-b8bf-864be1d8d5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:13.337845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d71a5-c892-417e-b8bf-864be1d8d5ef None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:13.353830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:13.466719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:13.467255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:13.467526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:13.467826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:13.469503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:13.573204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:13.574885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f615259-02a7-4033-882d-f156fa7b02f0 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:13.594523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 689/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:14.476762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a64adc-225b-48eb-907b-7c6ff3343b08 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:14.477394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a64adc-225b-48eb-907b-7c6ff3343b08 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:14.477394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a64adc-225b-48eb-907b-7c6ff3343b08 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:14.479172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 734/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:14.484988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a193-b8e6-4ebd-b7ee-882b0d159011 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:14.485516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a193-b8e6-4ebd-b7ee-882b0d159011 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:14.485516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3a193-b8e6-4ebd-b7ee-882b0d159011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:14.510076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:14.574250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9fb3a193-b8e6-4ebd-b7ee-882b0d159011 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:14.575567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9fb3a193-b8e6-4ebd-b7ee-882b0d159011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:14.683814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 690/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:14.841651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a64adc-225b-48eb-907b-7c6ff3343b08 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:14.843239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-09a64adc-225b-48eb-907b-7c6ff3343b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:14.966807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:15.006802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a64adc-225b-48eb-907b-7c6ff3343b08 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:15.139732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:15.140326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:15.140589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:15.140875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:15.142572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:15.226787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:15.243967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:15.246331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce796b31-938c-4580-8cc5-ff84087fd2bb service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:15.265953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 735/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:16.145426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc7e1-67dd-4746-9cb5-db2ce602e0ec None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:16.146012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc7e1-67dd-4746-9cb5-db2ce602e0ec None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:16.146012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdc7e1-67dd-4746-9cb5-db2ce602e0ec None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:16.147407 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 691/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:16.153025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2fc69-fb25-49f0-8a0d-5bb565d79869 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:16.153485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2fc69-fb25-49f0-8a0d-5bb565d79869 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:16.153582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2fc69-fb25-49f0-8a0d-5bb565d79869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:16.246820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-88c2fc69-fb25-49f0-8a0d-5bb565d79869 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:16.248099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88c2fc69-fb25-49f0-8a0d-5bb565d79869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:16.353257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 736/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:32:16.508100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc7e1-67dd-4746-9cb5-db2ce602e0ec None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:16.509592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-21cdc7e1-67dd-4746-9cb5-db2ce602e0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:16.604614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:16.673565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdc7e1-67dd-4746-9cb5-db2ce602e0ec None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:16.797817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:16.798454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:16.798734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:16.798999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:16.800748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:16.853482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:16.900872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:16.902519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55ca2-ee76-4e47-aab5-2efbf82fab6d service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:16.921480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 692/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:17.740117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed28a3-a45b-480e-8b6e-c8259fffc0f3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:17.740802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed28a3-a45b-480e-8b6e-c8259fffc0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:17.740802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed28a3-a45b-480e-8b6e-c8259fffc0f3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:17.742070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 737/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:17.747683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad961b3-447f-4ab4-bb81-ff92f157a157 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:17.748187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad961b3-447f-4ab4-bb81-ff92f157a157 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:17.748300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad961b3-447f-4ab4-bb81-ff92f157a157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:17.832112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4ad961b3-447f-4ab4-bb81-ff92f157a157 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:17.833621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4ad961b3-447f-4ab4-bb81-ff92f157a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:17.933289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 693/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:18.083850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed28a3-a45b-480e-8b6e-c8259fffc0f3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:18.085386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e8ed28a3-a45b-480e-8b6e-c8259fffc0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:18.136363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:18.241999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed28a3-a45b-480e-8b6e-c8259fffc0f3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:18.371379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:18.372044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:18.372319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:18.372588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:18.374246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:18.381564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:18.600418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:18.602223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a80b2-498f-4f9f-8073-e60724e3e9c8 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:18.623115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 738/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:19.423177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796900c-aba2-46a4-b587-0fa5f1e1f138 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:19.423750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796900c-aba2-46a4-b587-0fa5f1e1f138 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:19.423750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796900c-aba2-46a4-b587-0fa5f1e1f138 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:19.425184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 694/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:19.430192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc51be5-aeaa-45f7-9131-6d1f709b5b09 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:19.430644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc51be5-aeaa-45f7-9131-6d1f709b5b09 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:19.430749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc51be5-aeaa-45f7-9131-6d1f709b5b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:19.510077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-abc51be5-aeaa-45f7-9131-6d1f709b5b09 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:19.511417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-abc51be5-aeaa-45f7-9131-6d1f709b5b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:19.584722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:19.615186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 739/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:19.766894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796900c-aba2-46a4-b587-0fa5f1e1f138 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:19.768491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b796900c-aba2-46a4-b587-0fa5f1e1f138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:19.786542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:19.933787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796900c-aba2-46a4-b587-0fa5f1e1f138 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:20.025079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:20.057886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:20.059339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:20.059339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:20.059734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:20.061499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:20.156245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:20.157876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e58ec4-64a3-43c4-9b4f-1c63ee664d2d service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:20.177514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 695/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:20.987905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cfeb7-f988-4288-be09-9cf4fab22894 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:20.988486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cfeb7-f988-4288-be09-9cf4fab22894 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:20.988486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cfeb7-f988-4288-be09-9cf4fab22894 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:20.989760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 740/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:20.995048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0c99-4866-410e-9b68-5c399c9a0861 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:20.995505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0c99-4866-410e-9b68-5c399c9a0861 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:20.995608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0c99-4866-410e-9b68-5c399c9a0861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:21.211266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2c6a0c99-4866-410e-9b68-5c399c9a0861 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:21.212529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2c6a0c99-4866-410e-9b68-5c399c9a0861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:21.257478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:21.317592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 696/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:21.472176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cfeb7-f988-4288-be09-9cf4fab22894 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:21.473737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-646cfeb7-f988-4288-be09-9cf4fab22894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:21.628987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cfeb7-f988-4288-be09-9cf4fab22894 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:21.688418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:21.749246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:21.749804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:21.750102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:21.750378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:21.752041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:21.844964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:21.846579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe8542-35ed-4d91-a341-bb224f22c383 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:21.866191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 741/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:22.672199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95652789-ec52-42e2-98cc-494603114847 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:22.672758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95652789-ec52-42e2-98cc-494603114847 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:22.672758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95652789-ec52-42e2-98cc-494603114847 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:22.674132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 697/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:22.679723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53213aa3-00d7-4312-9d21-557c0b5bb42c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:22.680153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53213aa3-00d7-4312-9d21-557c0b5bb42c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:22.680312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53213aa3-00d7-4312-9d21-557c0b5bb42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:22.759656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-53213aa3-00d7-4312-9d21-557c0b5bb42c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:22.760949 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-53213aa3-00d7-4312-9d21-557c0b5bb42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:22.763048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:22.871552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 742/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:23.028194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95652789-ec52-42e2-98cc-494603114847 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:23.029832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-95652789-ec52-42e2-98cc-494603114847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:23.184000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95652789-ec52-42e2-98cc-494603114847 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:23.215662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:23.304415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:23.305042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:23.305289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:23.305557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:23.307280 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:23.405306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:23.406902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fe04c-fc25-46fe-ae53-ce74961f657f service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:23.424811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 698/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:25.383096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585ba3-d395-4bbf-aba2-e4b418dfeeec None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:25.383732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585ba3-d395-4bbf-aba2-e4b418dfeeec None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:25.383732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585ba3-d395-4bbf-aba2-e4b418dfeeec None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:25.385088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 743/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:25.391190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf898-ffed-4edc-a1d0-63f4f7724ba7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:25.391643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf898-ffed-4edc-a1d0-63f4f7724ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:25.391781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdf898-ffed-4edc-a1d0-63f4f7724ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:25.435031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:25.462698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a2cdf898-ffed-4edc-a1d0-63f4f7724ba7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:25.464594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a2cdf898-ffed-4edc-a1d0-63f4f7724ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:25.568289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 699/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:25.606801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585ba3-d395-4bbf-aba2-e4b418dfeeec None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:25.608384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-77585ba3-d395-4bbf-aba2-e4b418dfeeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:25.719142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585ba3-d395-4bbf-aba2-e4b418dfeeec None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:25.757166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:25.801834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:25.802452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:25.802724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:25.802996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:25.804645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:25.929889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:25.931240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde39df8-50f3-4453-92a8-afe96ef7cb31 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:25.950136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 744/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:28.127346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3694a-f116-4e90-a863-de1e28d7a21a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:28.128009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3694a-f116-4e90-a863-de1e28d7a21a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:28.128009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3694a-f116-4e90-a863-de1e28d7a21a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:28.129293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 700/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:28.134829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728020f-a35f-4799-92cf-73de22ecb329 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:28.135314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728020f-a35f-4799-92cf-73de22ecb329 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:28.135594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728020f-a35f-4799-92cf-73de22ecb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:28.207731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4728020f-a35f-4799-92cf-73de22ecb329 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:28.208999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4728020f-a35f-4799-92cf-73de22ecb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:28.222764 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:28.312855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 745/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:28.351274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3694a-f116-4e90-a863-de1e28d7a21a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:28.352870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-42e3694a-f116-4e90-a863-de1e28d7a21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:28.472336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3694a-f116-4e90-a863-de1e28d7a21a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:28.490636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:28.555179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:28.555838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:28.556119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:28.556390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:28.558076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:28.671505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:28.673013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edbba4-f17b-425d-b705-64d7ff24909f service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:28.693806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 701/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:31.132683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec723-367c-4576-84aa-a794c2791fa2 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:31.133298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec723-367c-4576-84aa-a794c2791fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:31.133298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec723-367c-4576-84aa-a794c2791fa2 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:31.134642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 746/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:31.140664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085fcfd-4c41-45b7-831c-bee79741c9fe None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:31.141124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085fcfd-4c41-45b7-831c-bee79741c9fe None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:31.141222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085fcfd-4c41-45b7-831c-bee79741c9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:31.194016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:31.215390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9085fcfd-4c41-45b7-831c-bee79741c9fe None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:31.216673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9085fcfd-4c41-45b7-831c-bee79741c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:31.319033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 702/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:31.358725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec723-367c-4576-84aa-a794c2791fa2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:31.360358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4eaec723-367c-4576-84aa-a794c2791fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:31.467568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec723-367c-4576-84aa-a794c2791fa2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:31.509279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:31.545159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:31.545780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:31.546048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:31.546314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:31.548097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:31.669676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:31.671416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be89778-fdca-456b-99e6-5cf1dc970527 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:31.692245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 747/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:33.870376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b1ed-6c2f-4c5a-9be8-4928ac4d1121 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:33.870947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b1ed-6c2f-4c5a-9be8-4928ac4d1121 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:33.870947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b1ed-6c2f-4c5a-9be8-4928ac4d1121 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:33.872538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 703/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:33.878209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cf39b-0bef-48fb-9009-902594c0bb39 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:33.878716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cf39b-0bef-48fb-9009-902594c0bb39 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:33.878811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cf39b-0bef-48fb-9009-902594c0bb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:33.953283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b46cf39b-0bef-48fb-9009-902594c0bb39 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:33.954409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b46cf39b-0bef-48fb-9009-902594c0bb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:33.977089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:34.064176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 748/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:34.104561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b1ed-6c2f-4c5a-9be8-4928ac4d1121 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:34.106152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dc37b1ed-6c2f-4c5a-9be8-4928ac4d1121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:34.222141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b1ed-6c2f-4c5a-9be8-4928ac4d1121 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:34.245151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:34.304129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:34.304808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:34.305083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:34.305352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:34.307064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:34.422610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:34.424114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d089c-ce47-4dce-a42f-476902700ebd service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:34.443540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 704/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:36.622579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413eab-7fa6-47b3-8099-154055cf7426 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:36.623178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413eab-7fa6-47b3-8099-154055cf7426 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:36.623178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413eab-7fa6-47b3-8099-154055cf7426 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:36.624576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 749/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:36.630940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f39e2-8852-4ccd-a5ee-6496db0850c5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:36.631607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f39e2-8852-4ccd-a5ee-6496db0850c5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:36.631607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f39e2-8852-4ccd-a5ee-6496db0850c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:36.717880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:36.720993 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3e6f39e2-8852-4ccd-a5ee-6496db0850c5 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:36.722990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3e6f39e2-8852-4ccd-a5ee-6496db0850c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:36.839259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 705/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:36.880101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413eab-7fa6-47b3-8099-154055cf7426 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:36.882041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-81413eab-7fa6-47b3-8099-154055cf7426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:37.001243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413eab-7fa6-47b3-8099-154055cf7426 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:37.052654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:37.078766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:37.079384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:37.079654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:37.079948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:37.081585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:37.190902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:37.192284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3be9a-b044-419c-a189-cd51da4161b0 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:37.210651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 750/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:39.274909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414f64-786f-4b6c-bbef-2b46f80bbd45 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:39.275489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414f64-786f-4b6c-bbef-2b46f80bbd45 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:39.275489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414f64-786f-4b6c-bbef-2b46f80bbd45 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:39.276859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 706/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:39.282666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40197bb1-ec28-4b46-bb99-2a62e8bc4afa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:39.283119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40197bb1-ec28-4b46-bb99-2a62e8bc4afa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:39.283240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40197bb1-ec28-4b46-bb99-2a62e8bc4afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:39.354572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-40197bb1-ec28-4b46-bb99-2a62e8bc4afa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:39.355870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-40197bb1-ec28-4b46-bb99-2a62e8bc4afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:39.385752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:39.458116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 751/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:39.498103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414f64-786f-4b6c-bbef-2b46f80bbd45 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:39.499637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f2414f64-786f-4b6c-bbef-2b46f80bbd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:39.614259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414f64-786f-4b6c-bbef-2b46f80bbd45 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:39.645462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:39.702568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:39.703221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:39.703499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:39.703792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:39.705473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:39.822880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:39.824377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e417-fa43-4fce-b701-8b3f0b019b70 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:39.843752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 707/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:41.947480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12daba-6aa0-4d73-9675-3b9170c992df None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:41.948075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12daba-6aa0-4d73-9675-3b9170c992df None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:41.948075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12daba-6aa0-4d73-9675-3b9170c992df None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:41.949507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 752/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:41.955620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceae02-728f-409b-b14c-5dcd96654073 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:41.956132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceae02-728f-409b-b14c-5dcd96654073 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:41.956238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ceae02-728f-409b-b14c-5dcd96654073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:42.028029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c1ceae02-728f-409b-b14c-5dcd96654073 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:42.029329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c1ceae02-728f-409b-b14c-5dcd96654073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:42.038812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:42.134058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 708/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:42.177117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12daba-6aa0-4d73-9675-3b9170c992df None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:42.178649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9a12daba-6aa0-4d73-9675-3b9170c992df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:42.290830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12daba-6aa0-4d73-9675-3b9170c992df None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:42.350250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:42.373053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:42.373682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:42.373958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:42.374226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:42.375919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:42.491740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:42.493217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b1d06-6852-4fca-a1db-0a6ea03088fd service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:42.511934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 753/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:44.614094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793e37f-41a3-481f-89e5-e7790edbd07d None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:44.614094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793e37f-41a3-481f-89e5-e7790edbd07d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:44.614575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793e37f-41a3-481f-89e5-e7790edbd07d None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:44.615703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 709/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:44.621770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412266e-de7a-4ce0-90de-33455a856d3d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:44.622235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412266e-de7a-4ce0-90de-33455a856d3d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:44.622345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412266e-de7a-4ce0-90de-33455a856d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:44.693171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3412266e-de7a-4ce0-90de-33455a856d3d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:44.694392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3412266e-de7a-4ce0-90de-33455a856d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:44.738229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:44.798284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 754/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:44.836149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793e37f-41a3-481f-89e5-e7790edbd07d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:44.837800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f793e37f-41a3-481f-89e5-e7790edbd07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:44.946745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793e37f-41a3-481f-89e5-e7790edbd07d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:44.985335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:45.028762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:45.029410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:45.029690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:45.029958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:45.031618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:45.150163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:45.151615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39e1f-d3c2-49c1-9f37-59a52a5c356b service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:45.170270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 710/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:47.226212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6692965-7ee0-40c4-975d-2d176c7e720d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:47.226790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6692965-7ee0-40c4-975d-2d176c7e720d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:47.226790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6692965-7ee0-40c4-975d-2d176c7e720d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:47.228296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 755/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:47.234079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14748531-f924-4ae3-8ad2-3815b7d68b84 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:47.234582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14748531-f924-4ae3-8ad2-3815b7d68b84 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:47.234582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14748531-f924-4ae3-8ad2-3815b7d68b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:47.303388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-14748531-f924-4ae3-8ad2-3815b7d68b84 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:47.304636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-14748531-f924-4ae3-8ad2-3815b7d68b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:47.323404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:47.408032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 711/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:47.445835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6692965-7ee0-40c4-975d-2d176c7e720d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:47.447591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f6692965-7ee0-40c4-975d-2d176c7e720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:47.559476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6692965-7ee0-40c4-975d-2d176c7e720d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:47.632329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:47.640471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:47.641102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:47.641381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:47.641650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:47.643322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:47.758529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:47.760010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884aed8-e2f5-4219-a4b8-cc63514e98e7 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:47.778691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 756/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:32:49.844925 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c45450-4a0a-49ad-9fe3-ecb7bcc1d223 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:49.845568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c45450-4a0a-49ad-9fe3-ecb7bcc1d223 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:49.845568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c45450-4a0a-49ad-9fe3-ecb7bcc1d223 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:49.846861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 712/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:32:49.852530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9932-97b3-4d04-a312-56d229f7545c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:49.852991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9932-97b3-4d04-a312-56d229f7545c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:49.853092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9932-97b3-4d04-a312-56d229f7545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:49.927249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f56d9932-97b3-4d04-a312-56d229f7545c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:32:49.928513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f56d9932-97b3-4d04-a312-56d229f7545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:49.979111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:50.029124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 757/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:32:50.066541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c45450-4a0a-49ad-9fe3-ecb7bcc1d223 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:32:50.068145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-33c45450-4a0a-49ad-9fe3-ecb7bcc1d223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:50.175585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c45450-4a0a-49ad-9fe3-ecb7bcc1d223 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:32:50.226053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:32:50.256829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:32:50.257378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:32:50.257655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:32:50.257924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:32:50.259565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:32:50.373659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:32:50.375110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87f9e3-4c8e-4435-9a09-a0f100ac79da service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:32:50.393124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 713/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:00.027165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=22 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:00.390795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=66 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:10.178243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56258a-c2eb-4d59-9dc1-17f60738260e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:10.178822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56258a-c2eb-4d59-9dc1-17f60738260e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:10.178822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56258a-c2eb-4d59-9dc1-17f60738260e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:10.180253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 758/1471] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:10.185782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd37457-4cb3-4386-a2a5-b9a8ee758a07 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:10.186286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd37457-4cb3-4386-a2a5-b9a8ee758a07 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:10.186286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd37457-4cb3-4386-a2a5-b9a8ee758a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:10.254657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7dd37457-4cb3-4386-a2a5-b9a8ee758a07 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:10.255901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7dd37457-4cb3-4386-a2a5-b9a8ee758a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:10.358655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 714/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:33:10.458328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6059c5a-f56b-45cb-a516-edf383caac3d None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:10.458616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6059c5a-f56b-45cb-a516-edf383caac3d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:10.458889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6059c5a-f56b-45cb-a516-edf383caac3d None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:10.460293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 759/1473] 10.222.0.45 () {54 vars in 966 bytes} [Mon Aug 1 16:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:10.465429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be010-ee61-4a5d-b4db-304450357506 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:10.465719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be010-ee61-4a5d-b4db-304450357506 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:10.465998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be010-ee61-4a5d-b4db-304450357506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:10.551275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-326be010-ee61-4a5d-b4db-304450357506 None None] MFA Rules not processed for user `288250b6b648409aa179b3213321b869`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:10.552538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-326be010-ee61-4a5d-b4db-304450357506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:10.573935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:10.630791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 715/1474] 10.222.0.45 () {58 vars in 1063 bytes} [Mon Aug 1 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:33:10.638657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6059c5a-f56b-45cb-a516-edf383caac3d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:10.640207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d6059c5a-f56b-45cb-a516-edf383caac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:10.749321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6059c5a-f56b-45cb-a516-edf383caac3d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:10.835739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:10.836416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:10.836684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:10.836945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:10.838573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:10.895131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:10.955094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:10.956617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3b692-0f2c-4954-8dba-5aea09044de8 service glance] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:10.975914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 760/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:33:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:13.084063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17779-fc35-4d3d-a55c-016496ed925f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:13.084637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17779-fc35-4d3d-a55c-016496ed925f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:13.084637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17779-fc35-4d3d-a55c-016496ed925f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:13.086079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 716/1476] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:13.092284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41db90-b8b3-4a3a-b8c3-4df23e30d421 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:13.092797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41db90-b8b3-4a3a-b8c3-4df23e30d421 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:13.092797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41db90-b8b3-4a3a-b8c3-4df23e30d421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:13.169517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bd41db90-b8b3-4a3a-b8c3-4df23e30d421 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:13.170752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bd41db90-b8b3-4a3a-b8c3-4df23e30d421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:13.296095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 761/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:33:13.319551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17779-fc35-4d3d-a55c-016496ed925f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:13.321130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4fb17779-fc35-4d3d-a55c-016496ed925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:13.456942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17779-fc35-4d3d-a55c-016496ed925f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:13.509369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:13.557306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:13.557905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:13.558303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:13.558575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:13.559292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:13.561111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:33:13.561111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:33:13.561111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:33:13.562252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008983e-45e9-4692-bba8-466f0afbf47d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:13.582763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 717/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Aug 1 16:33:13 2022] GET /identity/v3/projects => generated 2553 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:33:15.644667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971096c7-62dd-4d9e-bbfd-1af7c1534f0e None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:15.645277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971096c7-62dd-4d9e-bbfd-1af7c1534f0e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:15.645277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971096c7-62dd-4d9e-bbfd-1af7c1534f0e None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:15.646571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 762/1479] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:15.652891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed70cfd-bacd-4fd7-aad2-2eb0171ea03a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:15.652891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed70cfd-bacd-4fd7-aad2-2eb0171ea03a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:15.653540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed70cfd-bacd-4fd7-aad2-2eb0171ea03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:15.733823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7ed70cfd-bacd-4fd7-aad2-2eb0171ea03a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:15.735060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7ed70cfd-bacd-4fd7-aad2-2eb0171ea03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:15.837327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 718/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:33:15.890297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971096c7-62dd-4d9e-bbfd-1af7c1534f0e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:15.892074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-971096c7-62dd-4d9e-bbfd-1af7c1534f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:15.902785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:16.047959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971096c7-62dd-4d9e-bbfd-1af7c1534f0e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:16.133413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:16.171275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:16.171830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:16.172148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:16.172410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:16.174055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:16.268693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:16.270556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256e536-3b88-435a-84a4-4308bc6ff8c2 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:16.290685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 763/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:18.558485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec4022-e3a5-42c5-aec7-05e64e5a31ca None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:18.559105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec4022-e3a5-42c5-aec7-05e64e5a31ca None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:18.559215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec4022-e3a5-42c5-aec7-05e64e5a31ca None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:18.560601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 719/1482] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:18.566405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dc391-ed59-4311-a2f8-f093fe8c8678 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:18.566938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dc391-ed59-4311-a2f8-f093fe8c8678 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:18.566938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7dc391-ed59-4311-a2f8-f093fe8c8678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:18.638314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7c7dc391-ed59-4311-a2f8-f093fe8c8678 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:18.639564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7c7dc391-ed59-4311-a2f8-f093fe8c8678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:18.693123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:18.752463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 764/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:33:18.795253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec4022-e3a5-42c5-aec7-05e64e5a31ca None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:18.796823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cfec4022-e3a5-42c5-aec7-05e64e5a31ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:18.823016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:18.906666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec4022-e3a5-42c5-aec7-05e64e5a31ca None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:18.991625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:18.992309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:18.992591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:18.992864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:18.994530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:19.090656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:19.129793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:19.131252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba843cb-09f2-4fef-abc9-eb4fe99068cd service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:19.153476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 720/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:21.277157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26f77d-0eda-4963-8cac-9182838ca938 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:21.277932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26f77d-0eda-4963-8cac-9182838ca938 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:21.277932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26f77d-0eda-4963-8cac-9182838ca938 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:21.279118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 765/1485] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:21.285022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c71b5-5902-42d7-953e-5e16f4060bc3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:21.285554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c71b5-5902-42d7-953e-5e16f4060bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:21.285554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c71b5-5902-42d7-953e-5e16f4060bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:21.355550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2c8c71b5-5902-42d7-953e-5e16f4060bc3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:21.357088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2c8c71b5-5902-42d7-953e-5e16f4060bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:21.466837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 721/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:33:21.510439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26f77d-0eda-4963-8cac-9182838ca938 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:21.512832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-db26f77d-0eda-4963-8cac-9182838ca938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:21.622485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26f77d-0eda-4963-8cac-9182838ca938 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:21.699549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:21.701508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:21.702183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:21.702561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:21.702815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:21.705307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:21.821292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:21.822714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13484c2f-1249-46b9-8b96-76b4ad968754 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:21.841255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 766/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:23.957336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671767b-6bfb-4554-9ef2-4b4cfe9c6b3b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:23.957901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671767b-6bfb-4554-9ef2-4b4cfe9c6b3b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:23.957901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671767b-6bfb-4554-9ef2-4b4cfe9c6b3b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:23.959145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 722/1488] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:23.964964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22109e-b620-4ef4-b992-82cf572965ed None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:23.965524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22109e-b620-4ef4-b992-82cf572965ed None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:23.965524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22109e-b620-4ef4-b992-82cf572965ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:24.038279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7e22109e-b620-4ef4-b992-82cf572965ed None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:24.039981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7e22109e-b620-4ef4-b992-82cf572965ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:24.103781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:24.152305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 767/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:33:24.195572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671767b-6bfb-4554-9ef2-4b4cfe9c6b3b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:24.197477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2671767b-6bfb-4554-9ef2-4b4cfe9c6b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:24.243870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:24.320997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671767b-6bfb-4554-9ef2-4b4cfe9c6b3b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:24.409620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:24.410437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:24.410818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:24.411165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:24.413818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:24.505277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:24.541257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:24.542718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a5af3-5eed-4c4f-be33-8c96b45870a1 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:24.570705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 723/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:26.969519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcce22-a13a-48b3-af57-759d000e5d5c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:26.970083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcce22-a13a-48b3-af57-759d000e5d5c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:26.970083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcce22-a13a-48b3-af57-759d000e5d5c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:26.971340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 768/1491] 10.222.0.45 () {54 vars in 926 bytes} [Mon Aug 1 16:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:33:26.976861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dcf8a-ce00-4527-b6ee-b140443f86fe None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:26.977334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dcf8a-ce00-4527-b6ee-b140443f86fe None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:26.977437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dcf8a-ce00-4527-b6ee-b140443f86fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:27.046381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0c3dcf8a-ce00-4527-b6ee-b140443f86fe None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:33:27.047596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0c3dcf8a-ce00-4527-b6ee-b140443f86fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:27.155016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 724/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Aug 1 16:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:33:27.197581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcce22-a13a-48b3-af57-759d000e5d5c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:33:27.199194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1dfcce22-a13a-48b3-af57-759d000e5d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:27.315789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcce22-a13a-48b3-af57-759d000e5d5c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:33:27.398621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:33:27.399257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:33:27.399535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:33:27.399834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:33:27.401526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:33:27.411429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:27.523278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:33:27.524769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7e706-6286-4ae8-ad7e-836e6dc55abd service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:33:27.543822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 769/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:33:37.152387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=82 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:33:37.541120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=50 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:03.811574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:03.812628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:03.812628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:03.912000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:03.913865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:04.030616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 725/1494] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:35:04.042852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:04.043370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:04.043477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:04.131325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:04.133144 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:04.248531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 770/1495] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:35:04.255467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:04.257420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:04.270698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c56b0-60a8-4ef5-a8b3-5358e6774710 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:04.337632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:04.375292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:04.375954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:04.376230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:04.376499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:04.377264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:04.377566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:04.378592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:04.378592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:35:04.378592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:35:04.379475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:04.391311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 726/1496] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:04.400028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:04.409711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6a5fb-b0f3-492f-abd8-55610e0a63c7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:04.481221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:04.516318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:04.516941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:04.517231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:04.517492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:04.518148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:04.519942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:04.519942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:35:04.519942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:35:04.521136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:04.581208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 771/1497] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:35:04 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:04.585759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:04.587320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:04.622634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1d751-55f6-4ec2-8b55-5e503075f5de admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:04.734367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:04.734986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:04.735300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:04.735563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:04.736583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:04.737602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:04.737602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:35:04.737602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:35:04.738790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:04.788186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 727/1498] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:35:04 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:04.793003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:04.794672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:04.862793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015c52-3054-4052-8a09-189612d9e334 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:04.964116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:04.982267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:04.982894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:04.983167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:04.983435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:04.984155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:04.985190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:04.985190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:35:04.985190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:35:04.986454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:04.999495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 772/1499] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:35:05.003926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:05.005460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:05.017558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bda3e5-9665-4a35-b949-49a6c0534aed admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:05.048842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:05.128347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:05.128982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:05.129256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:05.129551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:05.130251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:05.131299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:05.131299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:35:05.131299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:35:05.132216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503a754-2d1f-4ba1-bca2-68f5185561f1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:05.146131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 728/1500] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:35:05.151365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:05.153173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:05.186709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6d1f1-beb5-4bd3-87b2-e07e97389f19 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:05.283572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:05.284273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:05.284541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:05.284793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] PATH_INFO: `/v3/projects/3d92a07157d046d199209201cb7243ea/users/dcd9ad1f64fa40c2922107b9badd719e/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:05.308951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d92a07157d046d199209201cb7243ea, user_id=dcd9ad1f64fa40c2922107b9badd719e, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:05.309992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:05.309992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:35:05.309992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:35:05.311958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9e4e5-3e3d-46be-bbfd-2c6d8cb57c8f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:05.348666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 773/1501] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:35:05 2022] PUT /identity/v3/projects/3d92a07157d046d199209201cb7243ea/users/dcd9ad1f64fa40c2922107b9badd719e/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:05.361812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e986e5-580d-4bee-9294-06a551c45a9f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:05.362100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e986e5-580d-4bee-9294-06a551c45a9f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:05.362397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e986e5-580d-4bee-9294-06a551c45a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:05.423998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-10e986e5-580d-4bee-9294-06a551c45a9f None None] MFA Rules not processed for user `dcd9ad1f64fa40c2922107b9badd719e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:05.425246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-10e986e5-580d-4bee-9294-06a551c45a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:05.520845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 729/1502] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:35:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:35:05.534751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e7912-93e8-41ac-9e1b-6a7fce5e444f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:05.534923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e7912-93e8-41ac-9e1b-6a7fce5e444f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:05.535225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e7912-93e8-41ac-9e1b-6a7fce5e444f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:05.536709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 774/1503] 10.222.0.45 () {54 vars in 964 bytes} [Mon Aug 1 16:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:05.542450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c2e20-f68b-467e-823f-ff77bbb260a3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:05.542924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c2e20-f68b-467e-823f-ff77bbb260a3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:05.543372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c2e20-f68b-467e-823f-ff77bbb260a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:05.576286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:05.625998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e36c2e20-f68b-467e-823f-ff77bbb260a3 None None] MFA Rules not processed for user `698e47aac57c48adb8836cec7226b039`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:05.627217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e36c2e20-f68b-467e-823f-ff77bbb260a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:05.705815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 730/1504] 10.222.0.45 () {58 vars in 1061 bytes} [Mon Aug 1 16:35:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:35:05.714160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e7912-93e8-41ac-9e1b-6a7fce5e444f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:05.715729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-960e7912-93e8-41ac-9e1b-6a7fce5e444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:05.769343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:05.827316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e7912-93e8-41ac-9e1b-6a7fce5e444f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:05.905041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:05.905665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:05.905942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:05.906215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:05.907872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:06.019312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:06.020822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:06.025451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:06.042693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 775/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:35:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:35:06.089821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a96bf-886e-4d40-861f-1c43e014064a None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:06.090304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a96bf-886e-4d40-861f-1c43e014064a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:06.090413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a96bf-886e-4d40-861f-1c43e014064a None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:06.091647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 731/1506] 10.222.0.45 () {52 vars in 830 bytes} [Mon Aug 1 16:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:06.095545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:06.096828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:06.216030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6f8c-2822-425f-8b3a-8f8c2dcc4abc service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:06.317962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:06.319065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:06.319402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:06.319671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] PATH_INFO: `/v3/users/dcd9ad1f64fa40c2922107b9badd719e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:06.333489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd9ad1f64fa40c2922107b9badd719e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:06.334457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:06.334457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:35:06.334457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:35:06.335641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:06.346146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:06.424750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd9ad1f64fa40c2922107b9badd719e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:06.425407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd9ad1f64fa40c2922107b9badd719e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:06.437277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd9ad1f64fa40c2922107b9badd719e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:06.464364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd9ad1f64fa40c2922107b9badd719e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:06.483911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd9ad1f64fa40c2922107b9badd719e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:06.490552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 776/1507] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:35:06 2022] DELETE /identity/v3/users/dcd9ad1f64fa40c2922107b9badd719e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:06.496322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:06.496600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:06.496874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:06.568462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:06.569696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:06.648474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:06.663379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 732/1508] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:35:06.674273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:06.675511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:06.787585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d7ad-280c-4e88-8da3-e21982c451fc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:06.867482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:06.888501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:06.889125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:06.889395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:06.889661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:06.891318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:07.037937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:07.040047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8241723-2dd1-4387-928d-fb3e37091d61 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:07.065170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 777/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:35:07.500127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:07.501393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:07.514338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fbb70-bd55-42ed-8d40-6249c8f0fb63 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:07.632478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:07.633150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:07.633429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:07.633718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] PATH_INFO: `/v3/projects/3d92a07157d046d199209201cb7243ea` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:07.646662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d92a07157d046d199209201cb7243ea)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:07.647728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:35:07.647728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:35:07.647728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:35:07.648971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:07.732969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d92a07157d046d199209201cb7243ea', 'request_id': 'req-de73ca9a-b4f5-4176-9a92-124850c78e15'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:07.741100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d92a07157d046d199209201cb7243ea', 'request_id': 'req-de73ca9a-b4f5-4176-9a92-124850c78e15'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:07.754529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d92a07157d046d199209201cb7243ea', 'request_id': 'req-de73ca9a-b4f5-4176-9a92-124850c78e15'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:07.791851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] The token cache is being invalidate because project 3d92a07157d046d199209201cb7243ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:35:07.792138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:07.792905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de73ca9a-b4f5-4176-9a92-124850c78e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:35:07.794319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 733/1510] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:35:07 2022] DELETE /identity/v3/projects/3d92a07157d046d199209201cb7243ea => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:17.061998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=62 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:17.789495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.071961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b0ab6-3bc5-4754-b6bb-626fe2c6e2fa None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.072647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b0ab6-3bc5-4754-b6bb-626fe2c6e2fa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.072647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b0ab6-3bc5-4754-b6bb-626fe2c6e2fa None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.073960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 778/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:51.079720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287657-d326-4fe2-b2a9-9ea988eb2648 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.080248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287657-d326-4fe2-b2a9-9ea988eb2648 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.080248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287657-d326-4fe2-b2a9-9ea988eb2648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.168859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-44287657-d326-4fe2-b2a9-9ea988eb2648 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:51.170099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-44287657-d326-4fe2-b2a9-9ea988eb2648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:51.288072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 734/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:36:51.450621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b0ab6-3bc5-4754-b6bb-626fe2c6e2fa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.453086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1e8b0ab6-3bc5-4754-b6bb-626fe2c6e2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:51.582325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b0ab6-3bc5-4754-b6bb-626fe2c6e2fa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.588343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe5352-3756-4e5b-8a66-40e9256563aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.589208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe5352-3756-4e5b-8a66-40e9256563aa service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.589490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe5352-3756-4e5b-8a66-40e9256563aa service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.589871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe5352-3756-4e5b-8a66-40e9256563aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.599079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe5352-3756-4e5b-8a66-40e9256563aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.600709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe5352-3756-4e5b-8a66-40e9256563aa service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.611898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 779/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:36:52.535265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0086-b6f4-4af8-aaf8-60eb55ec589b None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.535902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0086-b6f4-4af8-aaf8-60eb55ec589b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.535902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0086-b6f4-4af8-aaf8-60eb55ec589b None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.537226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 735/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:52.542800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ea28-7796-41eb-8a36-628e47255851 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.543257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ea28-7796-41eb-8a36-628e47255851 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.543363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ea28-7796-41eb-8a36-628e47255851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.564123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ea94ea28-7796-41eb-8a36-628e47255851 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:52.565320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ea94ea28-7796-41eb-8a36-628e47255851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:52.574357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 780/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:36:52.728948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0086-b6f4-4af8-aaf8-60eb55ec589b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.745534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0086-b6f4-4af8-aaf8-60eb55ec589b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.751401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361333c-5460-4380-92f5-e4af58736c76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.752112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361333c-5460-4380-92f5-e4af58736c76 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.752353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361333c-5460-4380-92f5-e4af58736c76 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.752624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361333c-5460-4380-92f5-e4af58736c76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.762222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361333c-5460-4380-92f5-e4af58736c76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.763890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361333c-5460-4380-92f5-e4af58736c76 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.774563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 736/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:36:52.905984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff648e5-d691-4427-b847-2068201063d9 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.906184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff648e5-d691-4427-b847-2068201063d9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.906456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff648e5-d691-4427-b847-2068201063d9 None None] PATH_INFO: `/v3` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.907919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 781/1517] 10.222.0.110 () {54 vars in 926 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:52.915277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0126722-7126-4a41-808b-7db54325a103 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.915565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0126722-7126-4a41-808b-7db54325a103 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.915875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0126722-7126-4a41-808b-7db54325a103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.981042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c0126722-7126-4a41-808b-7db54325a103 None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:52.982267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c0126722-7126-4a41-808b-7db54325a103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:53.002129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.059343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 737/1518] 10.222.0.110 () {58 vars in 1016 bytes} [Mon Aug 1 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:36:56.856746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f09c4f-b2f9-4e33-afae-200111f2dfc3 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.857431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f09c4f-b2f9-4e33-afae-200111f2dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.857431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f09c4f-b2f9-4e33-afae-200111f2dfc3 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.859104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 782/1519] 10.222.0.45 () {54 vars in 961 bytes} [Mon Aug 1 16:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:56.864394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897e592-2989-47a7-a86d-6626b4cd6772 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.865198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897e592-2989-47a7-a86d-6626b4cd6772 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.865370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897e592-2989-47a7-a86d-6626b4cd6772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.886358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2897e592-2989-47a7-a86d-6626b4cd6772 None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:56.887576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2897e592-2989-47a7-a86d-6626b4cd6772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:56.896601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 738/1520] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Aug 1 16:36:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:36:56.904647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f09c4f-b2f9-4e33-afae-200111f2dfc3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.919221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f09c4f-b2f9-4e33-afae-200111f2dfc3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.924794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ee9d1-0b0f-40a8-acf5-cf8b83efbf37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.925413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ee9d1-0b0f-40a8-acf5-cf8b83efbf37 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.925686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ee9d1-0b0f-40a8-acf5-cf8b83efbf37 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.925954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ee9d1-0b0f-40a8-acf5-cf8b83efbf37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.935371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ee9d1-0b0f-40a8-acf5-cf8b83efbf37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.937021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ee9d1-0b0f-40a8-acf5-cf8b83efbf37 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.946950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 783/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:04.045978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8865f-9948-4551-873c-4cae68ba756f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.046593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8865f-9948-4551-873c-4cae68ba756f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.046593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8865f-9948-4551-873c-4cae68ba756f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.047952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 739/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:04.053369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a931-8cfb-4fff-808e-32dbffd17700 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.053930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a931-8cfb-4fff-808e-32dbffd17700 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.053930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a931-8cfb-4fff-808e-32dbffd17700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.075538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1982a931-8cfb-4fff-808e-32dbffd17700 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:04.076805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1982a931-8cfb-4fff-808e-32dbffd17700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:04.085085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 784/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:37:04.243920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8865f-9948-4551-873c-4cae68ba756f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:04.259929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8865f-9948-4551-873c-4cae68ba756f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:04.265755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968bab4-9704-47e9-a520-aa0c201d7ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:04.266365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968bab4-9704-47e9-a520-aa0c201d7ea3 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.266643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968bab4-9704-47e9-a520-aa0c201d7ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.266913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968bab4-9704-47e9-a520-aa0c201d7ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.275842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968bab4-9704-47e9-a520-aa0c201d7ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:04.277457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968bab4-9704-47e9-a520-aa0c201d7ea3 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:04.287058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 740/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:05.122650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279dd1d-447d-4b12-9880-94536ec2feaf None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.123478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279dd1d-447d-4b12-9880-94536ec2feaf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.123478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279dd1d-447d-4b12-9880-94536ec2feaf None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.124631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 785/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:05.130291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcabc7-b10e-40e9-b47a-6201b74c3293 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.131003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcabc7-b10e-40e9-b47a-6201b74c3293 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.131203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcabc7-b10e-40e9-b47a-6201b74c3293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.152411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-47bcabc7-b10e-40e9-b47a-6201b74c3293 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:05.153639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-47bcabc7-b10e-40e9-b47a-6201b74c3293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:05.163119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 741/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:37:05.317772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279dd1d-447d-4b12-9880-94536ec2feaf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.333547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279dd1d-447d-4b12-9880-94536ec2feaf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.339810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e73b4-a9b2-4c29-9756-218b86895017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.340453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e73b4-a9b2-4c29-9756-218b86895017 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.340731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e73b4-a9b2-4c29-9756-218b86895017 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.341001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e73b4-a9b2-4c29-9756-218b86895017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.349784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e73b4-a9b2-4c29-9756-218b86895017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.351576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e73b4-a9b2-4c29-9756-218b86895017 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.364446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 786/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:09.707255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f785-c807-469d-a288-56a5c2756970 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.707894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f785-c807-469d-a288-56a5c2756970 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.707894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395f785-c807-469d-a288-56a5c2756970 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.709345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 742/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:09.716108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8964-bd7e-41be-9770-f2b1a80cc89d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.716663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8964-bd7e-41be-9770-f2b1a80cc89d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.716758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8964-bd7e-41be-9770-f2b1a80cc89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.738720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-284c8964-bd7e-41be-9770-f2b1a80cc89d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:09.739982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-284c8964-bd7e-41be-9770-f2b1a80cc89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:09.751283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 787/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:37:09.833466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395f785-c807-469d-a288-56a5c2756970 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.835218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4395f785-c807-469d-a288-56a5c2756970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:09.921400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395f785-c807-469d-a288-56a5c2756970 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.925959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266b990-b808-415b-86ac-05a958b51ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.926581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266b990-b808-415b-86ac-05a958b51ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.926837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266b990-b808-415b-86ac-05a958b51ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.927123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266b990-b808-415b-86ac-05a958b51ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.937578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266b990-b808-415b-86ac-05a958b51ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.938935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266b990-b808-415b-86ac-05a958b51ea2 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.948876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 743/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:12.003509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895728-b6d3-4d8c-a119-d9ed2daf4d75 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.003509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895728-b6d3-4d8c-a119-d9ed2daf4d75 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.003509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895728-b6d3-4d8c-a119-d9ed2daf4d75 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.003509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 788/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:12.005135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354043a4-ffc1-437c-a7ea-0d140b6e02be None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.005551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354043a4-ffc1-437c-a7ea-0d140b6e02be None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.005655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354043a4-ffc1-437c-a7ea-0d140b6e02be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.028694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-354043a4-ffc1-437c-a7ea-0d140b6e02be None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:12.029926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-354043a4-ffc1-437c-a7ea-0d140b6e02be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:12.038558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 744/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:37:12.076867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895728-b6d3-4d8c-a119-d9ed2daf4d75 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.078904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-73895728-b6d3-4d8c-a119-d9ed2daf4d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:12.093803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895728-b6d3-4d8c-a119-d9ed2daf4d75 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.097972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ee497-7dff-4e96-8f9b-78a0e1e33e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.098600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ee497-7dff-4e96-8f9b-78a0e1e33e04 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.098912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ee497-7dff-4e96-8f9b-78a0e1e33e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.099183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ee497-7dff-4e96-8f9b-78a0e1e33e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.109564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ee497-7dff-4e96-8f9b-78a0e1e33e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.111017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ee497-7dff-4e96-8f9b-78a0e1e33e04 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.120800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 789/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:15.897461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc1b7e-dea3-4da9-9c56-7c438ca6e11f None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.898110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc1b7e-dea3-4da9-9c56-7c438ca6e11f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.898226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc1b7e-dea3-4da9-9c56-7c438ca6e11f None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.899754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 745/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:15.905712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1765-6793-4709-80d1-c8f0bf1bf66a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.906194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1765-6793-4709-80d1-c8f0bf1bf66a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.906194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce1765-6793-4709-80d1-c8f0bf1bf66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.927349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5dce1765-6793-4709-80d1-c8f0bf1bf66a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:15.928622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5dce1765-6793-4709-80d1-c8f0bf1bf66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:15.938288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 790/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:37:16.250686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc1b7e-dea3-4da9-9c56-7c438ca6e11f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:16.252802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-adfc1b7e-dea3-4da9-9c56-7c438ca6e11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:16.319691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:16.334470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc1b7e-dea3-4da9-9c56-7c438ca6e11f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:16.338543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf16d68-41ef-4fc8-8790-7f410a03a6ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:16.339205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf16d68-41ef-4fc8-8790-7f410a03a6ed service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.339499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf16d68-41ef-4fc8-8790-7f410a03a6ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.339776 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf16d68-41ef-4fc8-8790-7f410a03a6ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.349719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf16d68-41ef-4fc8-8790-7f410a03a6ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:16.351278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf16d68-41ef-4fc8-8790-7f410a03a6ed service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:16.361596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 746/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:18.416074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba24f4-f81d-4169-bdf2-9386b2b36682 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.416802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba24f4-f81d-4169-bdf2-9386b2b36682 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.416802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba24f4-f81d-4169-bdf2-9386b2b36682 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.418002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 791/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:18.423446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eefa12-7722-4b4e-8679-f3b2895bcb56 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.423942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eefa12-7722-4b4e-8679-f3b2895bcb56 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.424053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eefa12-7722-4b4e-8679-f3b2895bcb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.445012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c4eefa12-7722-4b4e-8679-f3b2895bcb56 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:18.446240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c4eefa12-7722-4b4e-8679-f3b2895bcb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:18.455805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 747/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:37:18.777499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba24f4-f81d-4169-bdf2-9386b2b36682 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.779417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-89ba24f4-f81d-4169-bdf2-9386b2b36682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:18.787616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.795082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba24f4-f81d-4169-bdf2-9386b2b36682 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.798892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded97fb3-7bd4-45b9-b97c-248d27c8be45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.799537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97fb3-7bd4-45b9-b97c-248d27c8be45 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.799846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97fb3-7bd4-45b9-b97c-248d27c8be45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.800146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97fb3-7bd4-45b9-b97c-248d27c8be45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.810355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded97fb3-7bd4-45b9-b97c-248d27c8be45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.811838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded97fb3-7bd4-45b9-b97c-248d27c8be45 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.822636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 792/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:37:28.439463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=11 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.817537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:52.589103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d124e-28be-4334-a726-638ae31d23cb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.590187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d124e-28be-4334-a726-638ae31d23cb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.590187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d124e-28be-4334-a726-638ae31d23cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.623132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-422d124e-28be-4334-a726-638ae31d23cb None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:52.624370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-422d124e-28be-4334-a726-638ae31d23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:52.635543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 748/1540] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:38:52.637305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72e527-2ded-443b-b461-47c63a441f02 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.638151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72e527-2ded-443b-b461-47c63a441f02 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.638310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72e527-2ded-443b-b461-47c63a441f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.663903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.664263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.664458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.700405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ba72e527-2ded-443b-b461-47c63a441f02 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:52.702599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ba72e527-2ded-443b-b461-47c63a441f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:52.731246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 793/1541] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:52.736409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:52.737643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:52.746629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29560a3-5783-40b5-871c-b945242a8787 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.747123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29560a3-5783-40b5-871c-b945242a8787 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.747620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29560a3-5783-40b5-871c-b945242a8787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.766632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 749/1542] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 01 16:38:52.775736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:52.800604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c29560a3-5783-40b5-871c-b945242a8787 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:52.801157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c29560a3-5783-40b5-871c-b945242a8787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:52.802249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019eaf1-0da0-4d92-8da4-a75fa9d6b750 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:52.812318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:52.812318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.812318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.812318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.816196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:52.816720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:52.822273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:52.841382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 794/1543] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 01 16:38:52.852991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbd58-327a-469f-9e79-541ff6b0046a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.852991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbd58-327a-469f-9e79-541ff6b0046a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.853727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbd58-327a-469f-9e79-541ff6b0046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.858655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 750/1544] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:52.863845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:52.894814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44241801-32ad-4206-8e20-d1943c3b1f1b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:52.904438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:52.905149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:52.905421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:52.905726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:52.906387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:52.906687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:52.912882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:52.923593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cb8fbd58-327a-469f-9e79-541ff6b0046a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:52.931266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cb8fbd58-327a-469f-9e79-541ff6b0046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:52.942862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 751/1545] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:52.943749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:52.980780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbbb35-975a-4de3-8961-28755f764215 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:52.994795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 795/1546] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.005100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.005589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.006090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.026071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.028186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.028989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.029649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.031246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.035865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:38:53.035865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:38:53.035865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:38:53.037397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8616-e69e-40c9-8d5b-5b70cf7356ae admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.059892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.061304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.082910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 796/1547] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.083586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.124926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37555162-2c86-4cc9-acd3-76e02bed44a6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.135622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e71b19-7796-4314-8415-e6525bab81bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.136537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e71b19-7796-4314-8415-e6525bab81bd admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.136821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e71b19-7796-4314-8415-e6525bab81bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.137320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e71b19-7796-4314-8415-e6525bab81bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.138237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e71b19-7796-4314-8415-e6525bab81bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.142362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e71b19-7796-4314-8415-e6525bab81bd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.159335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 752/1548] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:52 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:53.164107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.164645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.164941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.211398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.218266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.230239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 797/1549] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:53.230239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b4b03-4454-43e8-b998-1f385c1302ed None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.230239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b4b03-4454-43e8-b998-1f385c1302ed None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.230239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b4b03-4454-43e8-b998-1f385c1302ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.257510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ac5b4b03-4454-43e8-b998-1f385c1302ed None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.258759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ac5b4b03-4454-43e8-b998-1f385c1302ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.360906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 798/1550] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.360906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511003-69cb-43cd-96c1-34ac4f34d2d4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.360906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511003-69cb-43cd-96c1-34ac4f34d2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.360906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511003-69cb-43cd-96c1-34ac4f34d2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.367892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 753/1551] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.367892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.377916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792d222-05f6-48ad-b6c5-52f795003c6e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.383160 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.383940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.384266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.384624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.385409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.387685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.393655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-47511003-69cb-43cd-96c1-34ac4f34d2d4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.395595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-47511003-69cb-43cd-96c1-34ac4f34d2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.408035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 799/1552] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.409925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751de72-cffe-4992-b20d-b9abf6a255d4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.410343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751de72-cffe-4992-b20d-b9abf6a255d4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.410727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751de72-cffe-4992-b20d-b9abf6a255d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.425116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 754/1553] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:53.425116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.442738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05fe7-f1a1-4bc6-a6c2-7ed359c103d0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.447941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3751de72-cffe-4992-b20d-b9abf6a255d4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.449173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3751de72-cffe-4992-b20d-b9abf6a255d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.453625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.454976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.455974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.456472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.457765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.465425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 800/1554] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.465678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.465766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd842c8-f6dd-4ce7-8d0b-3409aa105b56 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.465766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd842c8-f6dd-4ce7-8d0b-3409aa105b56 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.465766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd842c8-f6dd-4ce7-8d0b-3409aa105b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.498304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9fd842c8-f6dd-4ce7-8d0b-3409aa105b56 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.500465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9fd842c8-f6dd-4ce7-8d0b-3409aa105b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.500963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 755/1555] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/users => generated 288 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:53.501866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.514210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9623b99-b439-44d1-b63b-46445082f764 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.516259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 801/1556] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.518470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.518952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.519429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.524314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.525637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.526215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.526825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.528348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:53.528966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.532256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.547215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.548607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.552063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 756/1557] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:53.553656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.562441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 802/1558] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.562441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.568954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9879f98-3220-429d-b077-50bb4840ec61 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.572196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f97b6-57bf-4cac-9fa5-2a508de5efa4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.574096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbcf6e-2379-47d5-abf9-f16ee427be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.574671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbcf6e-2379-47d5-abf9-f16ee427be85 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.574945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbcf6e-2379-47d5-abf9-f16ee427be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.575215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbcf6e-2379-47d5-abf9-f16ee427be85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.575953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbcf6e-2379-47d5-abf9-f16ee427be85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.577585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbcf6e-2379-47d5-abf9-f16ee427be85 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.580422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.581197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.581678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.582158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.583459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:53.584025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.587116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d728401-0af3-4891-a32d-c73405134683 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.592734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 803/1559] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:53.596533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.596533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.596533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.608617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 757/1560] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:53.611847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513da5-5648-4a09-9ca0-876b07e79e92 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.612328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513da5-5648-4a09-9ca0-876b07e79e92 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.612798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513da5-5648-4a09-9ca0-876b07e79e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.628786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.630673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.636999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4e513da5-5648-4a09-9ca0-876b07e79e92 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.638252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4e513da5-5648-4a09-9ca0-876b07e79e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.651856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 804/1561] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.651856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.652561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 758/1562] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.652561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.652561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.652561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.669188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962950d-56ee-4ff0-bf5b-a376b2531dff None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.671560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.672852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:53.678702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.680426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.681177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.681643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.686186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 759/1563] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:53.686186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.686522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.686522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.692671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f6fdd-439c-49de-bcdf-3b8fa38db70a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.697722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.698361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.698640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.699078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.699801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.701757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.705725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 805/1564] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:53.707629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.719606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:53.725208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf644d59-56df-4622-85e3-b3dd7a204b08 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.732551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.733281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.733576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.733834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.734553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:53.734840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.735919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:38:53.735919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:38:53.735919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:38:53.736815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.743107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 760/1565] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:53.744380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.753747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 806/1566] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:53.754604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.770285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273d37-1865-40d4-ac68-7fbca52a7516 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.836676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65ac53-9808-4cdf-bdc0-2f47cacd8ed7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.843320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:53.846537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.847730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.848263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.848734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.850004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.853129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.864598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2f571-6e78-488d-bf93-00d3297b5b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.865388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f571-6e78-488d-bf93-00d3297b5b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.865706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f571-6e78-488d-bf93-00d3297b5b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.866130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f571-6e78-488d-bf93-00d3297b5b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.866963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2f571-6e78-488d-bf93-00d3297b5b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.869066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2f571-6e78-488d-bf93-00d3297b5b42 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.872548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 761/1567] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:53.874070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.891876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf82673-a261-473c-9cc7-db746445e064 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.899910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:53.900559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.901038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.901201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.901820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:53.902123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:53.903916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:53.909030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 807/1568] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:53.911180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:53.911677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:53.912182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:53.930399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 762/1569] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:53.930399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:53.932750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e4245-5762-4a53-8032-9254da145bcd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:53.945155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:53.946682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:54.016067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.017114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.017580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.018068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.018432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 808/1570] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:54.019609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.025966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:54.026747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.030958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.034251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab26a07-3be1-4953-adfc-e1a27266872c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.042702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.043794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.044258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.044720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.046102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.049241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.062248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 763/1571] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:54.062248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.074861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207c3d7-f6f8-4a48-a62d-79c5f67d40b8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.079544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 809/1572] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:54.079544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.087393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.088803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.089551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.090112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.091579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.095481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.097840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da829e24-e4b0-49c7-9fd8-9621d61c34d8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.105874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.106856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.107304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.107787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.108938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.112352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.136381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 764/1573] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/users => generated 286 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.137910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.152851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292a917-96ed-4908-8b08-54a466807cbc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.159997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.161049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.161430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.161875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 810/1574] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.162046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] PATH_INFO: `/v3/projects/c6d1d41a90df4eb8ae7b9a3934629614/users/7a310f36f74e4eaabd9f32220feb274a/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.163354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.191729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d1d41a90df4eb8ae7b9a3934629614, user_id=7a310f36f74e4eaabd9f32220feb274a, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.193137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:38:54.193137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:38:54.193137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:38:54.196756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.217758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 765/1575] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:54 2022] PUT /identity/v3/projects/c6d1d41a90df4eb8ae7b9a3934629614/users/7a310f36f74e4eaabd9f32220feb274a/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:54.219337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.259221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad968f58-72ad-4757-94e7-40a6e84d5568 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.262275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:54.308922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf4cbd-d45b-45b3-9c0f-d31a0d525198 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.316761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:54.317145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.318362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.318934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.319737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.321315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.325568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.341448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.342099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.342568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.342815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.343551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.344660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:38:54.344660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:38:54.344660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:38:54.346006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.372187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 811/1576] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.378117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 766/1577] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.378117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.378516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.387043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee603b51-324e-4763-9a8d-eac1649a5458 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.460126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.460870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.461249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.461649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.462751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:54.463260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.465257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.466174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173a7b2-0ed9-4cf4-8c76-daf2f0f33b16 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.472981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.474857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.475403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.475940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.476778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.480574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.482731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 812/1578] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:54.483835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.503000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb8bb-e691-4226-9377-8123e871508a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.509436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.510357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.510879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.511241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] PATH_INFO: `/v3/projects/1b66d46c3e7d42b88f2dbdc01232fb03/users/4b0aed0278344119aae6906f4f52b1e5/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.529286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 767/1579] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.529286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.537518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b66d46c3e7d42b88f2dbdc01232fb03, user_id=4b0aed0278344119aae6906f4f52b1e5, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.542269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.563174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 813/1580] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:54 2022] PUT /identity/v3/projects/1b66d46c3e7d42b88f2dbdc01232fb03/users/4b0aed0278344119aae6906f4f52b1e5/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:54.564443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.622848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f6063-a677-44ea-98c0-82f0ca27ce38 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.653199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b980-aedc-4faa-9a78-4392685bd7ee admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.659534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d00573-31e8-4731-8a86-590f99be239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.660322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00573-31e8-4731-8a86-590f99be239c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.660749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00573-31e8-4731-8a86-590f99be239c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.661186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00573-31e8-4731-8a86-590f99be239c admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.662150 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d00573-31e8-4731-8a86-590f99be239c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.664974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d00573-31e8-4731-8a86-590f99be239c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.691111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:54.699289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 814/1581] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.701720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.702247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.702714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.711430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.712094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.712376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.712755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.713529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.715375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.731258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 768/1582] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:54.731258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.737440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:54.739024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2b5a7-8c41-4201-aa79-e140b72a7015 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.739478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:54.744451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.745235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.745933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.746386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.747676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.750828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.755320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 815/1583] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:54.756871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.765358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 769/1584] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:54.766314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.775983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83979138-4496-42c5-90ab-984b8c10b08b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.777048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bbcc1-4b54-4e1a-932b-7a1f3ab90ab3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.781848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.782483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.782766 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.783039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.783711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.783904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2d9df-b9b0-42b6-ae67-fd0a0766de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.784860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d9df-b9b0-42b6-ae67-fd0a0766de53 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.785395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d9df-b9b0-42b6-ae67-fd0a0766de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.785696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.785879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d9df-b9b0-42b6-ae67-fd0a0766de53 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.787090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2d9df-b9b0-42b6-ae67-fd0a0766de53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.790888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2d9df-b9b0-42b6-ae67-fd0a0766de53 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.819055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 770/1585] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.819988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.831817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 816/1586] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/users => generated 297 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.831817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.831817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.832187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.865720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:54.868600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:54.875951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:54.902435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189eb34-2316-4007-95ec-720b863d4e3f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.909457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.910108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.910381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.910667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.911335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.913643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.938886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 771/1587] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:54.939745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.953817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 817/1588] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:54.957208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e64c3f-9565-450e-a9dc-b0aef89c0b18 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.957515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.966541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.967552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.968105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.968594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.968931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d0cf1-21a8-4eb7-b497-837e134e955a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:54.969923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.973088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.973771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:54.974518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:54.974895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:54.975529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:54.976727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:54.979996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:54.992578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 772/1589] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:54.993616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:54.997917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 818/1590] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.001392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.009044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d1df0-2e37-496b-8f0d-19b2bff09142 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.016489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.017392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.017796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.018166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.019282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.021520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.052211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 773/1591] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.052211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.064332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71df68-bf02-4001-810a-80d9d3d4a5bb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.069423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.070387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.070717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.071037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.072026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.074510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.090743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56ed7-5a2f-44bd-b70e-eea1c73c3907 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.098321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.099327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.099711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.100515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.108085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 774/1592] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:55 2022] POST /identity/v3/users => generated 287 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:55.108085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.115481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.117029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.119154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e531d2-20e9-42d8-a133-d4b67766bd76 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.125541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.126177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.126444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.126706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.127411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.129205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.129842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 819/1593] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:38:55.131324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.142118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 775/1594] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.145331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.145666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a946d-09bd-4783-a1c5-9baf153d73fb service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.153824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.154772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.155129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.155597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.156867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939fc5d-712e-43ad-ab53-2dcf31e4e803 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.157085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.160561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.160776 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.161480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.161818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.162152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.173951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.175561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.180687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 820/1595] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.182211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.191547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 776/1596] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:38:55.192384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.201759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e910f-5afd-4292-b0f0-c4544dc20947 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.204438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a525cb1-dc3f-4a54-9073-d41504130397 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.210028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.210329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.210938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.211277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.211454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.211602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] PATH_INFO: `/v3/projects/d825d47bf1a64c19b117d7a62bb69fc0/users/d7f782d61baf46eb9ae770372adc023b/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.211839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.212372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.213596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.217027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.232903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d825d47bf1a64c19b117d7a62bb69fc0, user_id=d7f782d61baf46eb9ae770372adc023b, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.235605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.245970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 821/1597] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.245970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.258090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899033e-a2c1-44bc-b6a0-7a931499c1df admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.267455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 777/1598] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:55 2022] PUT /identity/v3/projects/d825d47bf1a64c19b117d7a62bb69fc0/users/d7f782d61baf46eb9ae770372adc023b/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:55.269840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.291984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:55.352573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.353291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.353629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.354052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] PATH_INFO: `/v3/projects/11fb0fc60eac4bf3bbed71ccd34fb778/users/7b2dc38f3afd466c8660752a24ad02dd/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.378024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b050d7-2e67-4206-ac31-ba8f7e2079ab admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.383006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fb0fc60eac4bf3bbed71ccd34fb778, user_id=7b2dc38f3afd466c8660752a24ad02dd, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.385268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.386038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.386403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.387217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.387362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.387481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.389484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.404253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 778/1600] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.405595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.407322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 822/1600] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:55 2022] PUT /identity/v3/projects/11fb0fc60eac4bf3bbed71ccd34fb778/users/7b2dc38f3afd466c8660752a24ad02dd/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:55.408532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.416971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:55.417382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d932a9c-75e5-498c-87f2-db6210876a91 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.525502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79b6ca-037c-4064-8f79-9e7d81a27e1f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.526128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490a0d5-3a0c-4e7a-a24c-957c6a71101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.527346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490a0d5-3a0c-4e7a-a24c-957c6a71101f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.527884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490a0d5-3a0c-4e7a-a24c-957c6a71101f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.528376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490a0d5-3a0c-4e7a-a24c-957c6a71101f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.530716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490a0d5-3a0c-4e7a-a24c-957c6a71101f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.533287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490a0d5-3a0c-4e7a-a24c-957c6a71101f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.533741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.534745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.535157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.535565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.536782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.539608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.564634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 823/1602] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.564857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 779/1602] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.564990 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.565084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba8b5d-0142-4f7a-8653-e99512b0841b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.565084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba8b5d-0142-4f7a-8653-e99512b0841b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.565084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba8b5d-0142-4f7a-8653-e99512b0841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.584552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00ff4-3536-48d2-b530-2f6dd5fd5a32 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.594408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.595797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.596457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.596803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] PATH_INFO: `/v3/projects/2f46e472f8614c5e847ceb225cc9bff1/users/1ac7d3acf1204a9f98f9ee14c2b46a5e/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.613213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-92ba8b5d-0142-4f7a-8653-e99512b0841b None None] MFA Rules not processed for user `d7f782d61baf46eb9ae770372adc023b`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:55.615253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-92ba8b5d-0142-4f7a-8653-e99512b0841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:55.632647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f46e472f8614c5e847ceb225cc9bff1, user_id=1ac7d3acf1204a9f98f9ee14c2b46a5e, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.638255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.671202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 824/1603] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:55 2022] PUT /identity/v3/projects/2f46e472f8614c5e847ceb225cc9bff1/users/1ac7d3acf1204a9f98f9ee14c2b46a5e/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:55.681176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.751273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 780/1604] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:55.753267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.753723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.754007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.808983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] MFA Rules not processed for user `7b2dc38f3afd466c8660752a24ad02dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:55.811018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:55.830843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee93fa-332d-477c-99f2-ea26bea022b4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.859282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.859968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.860339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.860502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.861242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.862934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.880534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:55.889875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 825/1605] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:55.890515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:55.926884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8e613-ca9f-411e-99fb-8d026c44ead8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:55.948199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012bb51-80ca-4c1f-895f-7fd90d28f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:55.948627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012bb51-80ca-4c1f-895f-7fd90d28f01c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:55.948905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012bb51-80ca-4c1f-895f-7fd90d28f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:55.949254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012bb51-80ca-4c1f-895f-7fd90d28f01c admin admin] PATH_INFO: `/v3/projects/dbd6814591334db58a37092eb04729fb/users/4294d6dff5054da7afcb37b8b28f1a01/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:55.986974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012bb51-80ca-4c1f-895f-7fd90d28f01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd6814591334db58a37092eb04729fb, user_id=4294d6dff5054da7afcb37b8b28f1a01, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:55.989716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012bb51-80ca-4c1f-895f-7fd90d28f01c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:55.999446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 781/1606] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:56.000392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.032931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 826/1607] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:55 2022] PUT /identity/v3/projects/dbd6814591334db58a37092eb04729fb/users/4294d6dff5054da7afcb37b8b28f1a01/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:56.035498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.035975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.039001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.042583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cc888-7ef4-4499-a07b-e52ad89650a4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.083271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:56.084756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:56.144148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:56.166864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 827/1608] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:56.167895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.179565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318af3f7-f503-4ddc-a8b0-78ed59903766 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.185470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:56.186120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.186396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.186667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.187407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:56.188999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:56.189466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:56.190605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.191228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.192233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] PATH_INFO: `/v3/projects/4c1c9729df694f46987e6dd30a09d5d2/users/76c1fab92b854cd8937cc80818159191/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.224844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1c9729df694f46987e6dd30a09d5d2, user_id=76c1fab92b854cd8937cc80818159191, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:56.235154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 828/1609] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:56.236166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.237391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:56.270937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 782/1610] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:55 2022] PUT /identity/v3/projects/4c1c9729df694f46987e6dd30a09d5d2/users/76c1fab92b854cd8937cc80818159191/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:56.274572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.358063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e818c1-9b83-4d8f-bb0c-15ad7702a118 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.466134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58b68-769d-44af-882d-f32f897891f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:56.467341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58b68-769d-44af-882d-f32f897891f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.467554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58b68-769d-44af-882d-f32f897891f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.467878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58b68-769d-44af-882d-f32f897891f2 admin admin] PATH_INFO: `/v3/projects/51b7e06f5b96485e8449852d67908bc1/users/7ebe8558580445f1806be8be7398e2c7/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.487875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be58b68-769d-44af-882d-f32f897891f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b7e06f5b96485e8449852d67908bc1, user_id=7ebe8558580445f1806be8be7398e2c7, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:56.491235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be58b68-769d-44af-882d-f32f897891f2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:56.505134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564f775-fc48-48d5-9bd0-ab37d5ff12d7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.516248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:56.516903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.517175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.517440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.519406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:56.524876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 829/1611] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:56 2022] PUT /identity/v3/projects/51b7e06f5b96485e8449852d67908bc1/users/7ebe8558580445f1806be8be7398e2c7/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:56.527840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.528926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:56.532427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.532891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.592520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:56.593954 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:56.634152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 783/1612] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/projects => generated 367 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:56.635959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.672798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:56.690982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 830/1613] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:56.692185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.707147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10706d3-e9ce-447b-93fd-2363264bf477 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.743286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:56.798043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958f807-e84e-4c31-afed-80d4adad686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:56.799626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f807-e84e-4c31-afed-80d4adad686e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.799626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f807-e84e-4c31-afed-80d4adad686e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.799843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f807-e84e-4c31-afed-80d4adad686e admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.800470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f807-e84e-4c31-afed-80d4adad686e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:56.802668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f807-e84e-4c31-afed-80d4adad686e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:56.811984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926248c4-4989-4b72-aba9-d59c71737cba admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.826407 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:56.828030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.828518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.829368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.842167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 831/1614] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:56.843927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed936b8-98e0-47bd-af44-b8c8b92918a7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.844377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed936b8-98e0-47bd-af44-b8c8b92918a7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.844507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed936b8-98e0-47bd-af44-b8c8b92918a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.863745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:56.871334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:56.872461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-aed936b8-98e0-47bd-af44-b8c8b92918a7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:56.873817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aed936b8-98e0-47bd-af44-b8c8b92918a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:56.892481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 832/1615] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:56.892481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:56.892481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:56.892481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:56.923651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 784/1616] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:38:56.927732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:56.975957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bface462-6c6c-4f05-b130-0f18f9e18103 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:56.999217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.001375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.003258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.006011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.007265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.008817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] MFA Rules not processed for user `7ebe8558580445f1806be8be7398e2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:57.009397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.012080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:57.057090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 785/1617] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:57.060422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.120261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0d5e-6349-4620-a747-5866fbc7cf33 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.138781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.140022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.140963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.141731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.144516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.145574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 833/1618] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:57.146829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.151297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.192226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6a440-6f35-4fab-bea0-6d97d7408802 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.201664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.202787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.203306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.203830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.212094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 786/1619] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:57.214507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.232151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.234265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.248308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 834/1620] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:38:57.249731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.265544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75921c-3457-4af8-825b-99ad174724b7 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.268943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736cab2-ad91-4bf6-bba7-6e886b45f5b1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.275583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd638c-01fe-4bde-8c60-096298b94b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.276938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd638c-01fe-4bde-8c60-096298b94b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.277505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd638c-01fe-4bde-8c60-096298b94b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.278056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd638c-01fe-4bde-8c60-096298b94b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.279529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd638c-01fe-4bde-8c60-096298b94b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.281658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd638c-01fe-4bde-8c60-096298b94b50 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.285878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.288899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.288899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.289883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.302460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 835/1621] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:57.305364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff78f2-4192-4ac3-8021-95da9eafbee0 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.305977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff78f2-4192-4ac3-8021-95da9eafbee0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.306485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff78f2-4192-4ac3-8021-95da9eafbee0 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.313724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 836/1622] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 16:38:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:57.315075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff78f2-4192-4ac3-8021-95da9eafbee0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.320453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.323963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.330588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff78f2-4192-4ac3-8021-95da9eafbee0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.337397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157ffb8-9664-4885-bd85-4965980fbdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.338487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157ffb8-9664-4885-bd85-4965980fbdee admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.339010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157ffb8-9664-4885-bd85-4965980fbdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.339842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157ffb8-9664-4885-bd85-4965980fbdee admin admin] PATH_INFO: `/v3/projects/8d3a811a1b0f43999fbfa1c256e95da6/users/0d338786949c4ffdbb92863c8c863643/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.361331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 787/1623] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:38:57.361331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.383649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157ffb8-9664-4885-bd85-4965980fbdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3a811a1b0f43999fbfa1c256e95da6, user_id=0d338786949c4ffdbb92863c8c863643, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.389371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157ffb8-9664-4885-bd85-4965980fbdee admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.396151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acaa6c-524d-4ab6-b48c-e5626b0e4277 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.413279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.414795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.415396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.416624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] PATH_INFO: `/v3/projects/40d6385d2fd94e4e99c3e79385c23013/users/29aa2e8e6adc41ac804ab41931250080/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.422943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 837/1624] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:57 2022] PUT /identity/v3/projects/8d3a811a1b0f43999fbfa1c256e95da6/users/0d338786949c4ffdbb92863c8c863643/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:57.424923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.427220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.427793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.483130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d6385d2fd94e4e99c3e79385c23013, user_id=29aa2e8e6adc41ac804ab41931250080, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.497541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.524208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:57.535507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:57.536760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:57.551461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 788/1625] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:57 2022] PUT /identity/v3/projects/40d6385d2fd94e4e99c3e79385c23013/users/29aa2e8e6adc41ac804ab41931250080/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:57.553091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.690929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74de95-164b-4319-8e7e-71c39a55ab75 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.708875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.710011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.710317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.710629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.712102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.714022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.725212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 838/1626] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 16:38:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:57.743056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.769492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 789/1627] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:57.769844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca83c02-adb4-4cbf-bcf1-8bab2e2ce4cf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.771666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.789352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.790028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.790317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.791263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.791619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.794898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.816526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd184a3-7a1c-4a78-ace5-9fe3d25234bc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.840968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 839/1628] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:38:57.842858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80988f-b276-4e75-8458-16829688a04c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.847510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80988f-b276-4e75-8458-16829688a04c service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.847510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80988f-b276-4e75-8458-16829688a04c service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.847510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80988f-b276-4e75-8458-16829688a04c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.849483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.878746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9f142-d32a-4498-905c-e8f1e12023b8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:57.890771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:57.891510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:57.893964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:57.894378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] PATH_INFO: `/v3/projects/8d3a811a1b0f43999fbfa1c256e95da6/users/0d338786949c4ffdbb92863c8c863643/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:57.899237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3a811a1b0f43999fbfa1c256e95da6, user_id=0d338786949c4ffdbb92863c8c863643, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.902332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:57.945440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 840/1629] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:57 2022] PUT /identity/v3/projects/8d3a811a1b0f43999fbfa1c256e95da6/users/0d338786949c4ffdbb92863c8c863643/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:57.945440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:57.993990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80988f-b276-4e75-8458-16829688a04c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:57.995669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80988f-b276-4e75-8458-16829688a04c service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:58.020189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 790/1630] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:38:58.025597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:58.026008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:58.026145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:58.052676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:58.099271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] MFA Rules not processed for user `0d338786949c4ffdbb92863c8c863643`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:58.100933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:58.110569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d524bf2-823c-48d2-8707-5cd829bc2b5b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:58.120862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766bdcc-5f6e-4aaa-8c05-dd5869d1ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:58.122810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766bdcc-5f6e-4aaa-8c05-dd5869d1ed8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:58.123158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766bdcc-5f6e-4aaa-8c05-dd5869d1ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:58.123441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766bdcc-5f6e-4aaa-8c05-dd5869d1ed8c admin admin] PATH_INFO: `/v3/projects/40d6385d2fd94e4e99c3e79385c23013/users/29aa2e8e6adc41ac804ab41931250080/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:58.129087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766bdcc-5f6e-4aaa-8c05-dd5869d1ed8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d6385d2fd94e4e99c3e79385c23013, user_id=29aa2e8e6adc41ac804ab41931250080, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:58.136426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766bdcc-5f6e-4aaa-8c05-dd5869d1ed8c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:58.167334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 841/1631] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:38:57 2022] PUT /identity/v3/projects/40d6385d2fd94e4e99c3e79385c23013/users/29aa2e8e6adc41ac804ab41931250080/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:58.179316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:58.179603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:58.179944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:58.253073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] MFA Rules not processed for user `29aa2e8e6adc41ac804ab41931250080`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:58.255179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 791/1632] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:58.261741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:58.268687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:58.478169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7c240-4acc-4ee2-abc3-ced0954dd8c1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:58.505578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:58.549871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 842/1633] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:38:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:58.567083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:58.610376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4426475-35d7-42dc-96cc-e3e15ce068dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:58.611551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4426475-35d7-42dc-96cc-e3e15ce068dc service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:58.612051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4426475-35d7-42dc-96cc-e3e15ce068dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:58.612507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4426475-35d7-42dc-96cc-e3e15ce068dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:58.799182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92dada-e71f-4892-a0a3-9826ad21f7c6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:58.799182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f46d5-dd9d-4b76-b2c5-4157dc306f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:58.799182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f46d5-dd9d-4b76-b2c5-4157dc306f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:58.799182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f46d5-dd9d-4b76-b2c5-4157dc306f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:58.806624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f46d5-dd9d-4b76-b2c5-4157dc306f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:58.819890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:58.820647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f46d5-dd9d-4b76-b2c5-4157dc306f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:58.821690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f46d5-dd9d-4b76-b2c5-4157dc306f3a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:58.831793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4426475-35d7-42dc-96cc-e3e15ce068dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:58.843577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4426475-35d7-42dc-96cc-e3e15ce068dc service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:58.857540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 843/1634] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:38:58.894693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 792/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:38:59.411795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7b6ca-4e1f-4302-8dea-87d8896ae32a None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:59.413041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7b6ca-4e1f-4302-8dea-87d8896ae32a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:59.413266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7b6ca-4e1f-4302-8dea-87d8896ae32a None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:59.415854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 844/1636] 10.222.0.45 () {54 vars in 962 bytes} [Mon Aug 1 16:38:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:59.433747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8611a0f-f525-4f61-bad8-17c0a545c3ae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:59.434477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8611a0f-f525-4f61-bad8-17c0a545c3ae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:59.434870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8611a0f-f525-4f61-bad8-17c0a545c3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:59.477004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e8611a0f-f525-4f61-bad8-17c0a545c3ae None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:59.481391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8611a0f-f525-4f61-bad8-17c0a545c3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:59.702782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 793/1637] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Aug 1 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:38:59.719236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930a4c0-c56c-4e63-811d-4dcbd930b776 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:59.720716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930a4c0-c56c-4e63-811d-4dcbd930b776 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:59.721624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930a4c0-c56c-4e63-811d-4dcbd930b776 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:59.732309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 845/1638] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 16:38:59 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:38:59.745505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3f71-d9aa-4a52-92ef-94e3ea21e9db None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:59.746157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3f71-d9aa-4a52-92ef-94e3ea21e9db None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:59.746985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3f71-d9aa-4a52-92ef-94e3ea21e9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:59.803503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f9fc3f71-d9aa-4a52-92ef-94e3ea21e9db None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:59.807942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f9fc3f71-d9aa-4a52-92ef-94e3ea21e9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:59.844997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 794/1639] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:38:59.909062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930a4c0-c56c-4e63-811d-4dcbd930b776 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:59.923232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930a4c0-c56c-4e63-811d-4dcbd930b776 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:59.933558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace7a34-05ae-4b9f-88f6-86313ef2d0a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:59.934379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7a34-05ae-4b9f-88f6-86313ef2d0a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:59.934721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7a34-05ae-4b9f-88f6-86313ef2d0a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:59.935234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7a34-05ae-4b9f-88f6-86313ef2d0a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:59.958813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace7a34-05ae-4b9f-88f6-86313ef2d0a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:59.961237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace7a34-05ae-4b9f-88f6-86313ef2d0a2 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:59.980594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 846/1640] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:38:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:00.451951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea169c2d-57c6-4ddc-9a9d-0e6fa6234c98 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:00.451951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea169c2d-57c6-4ddc-9a9d-0e6fa6234c98 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:00.451951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea169c2d-57c6-4ddc-9a9d-0e6fa6234c98 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:00.451951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 795/1641] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 16:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:00.458702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:00.458702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:00.458702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:00.500961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea169c2d-57c6-4ddc-9a9d-0e6fa6234c98 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:00.542528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:00.546971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:00.608287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 847/1642] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:39:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:00.624116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:00.639867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5b90b-fcbe-4662-a2cf-4f5604b3740f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:00.641331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea169c2d-57c6-4ddc-9a9d-0e6fa6234c98 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:00.651594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:00.652310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:00.652644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:00.653396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:00.657307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:00.658328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:00.658805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:00.659187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] PATH_INFO: `/v3/users/d7f782d61baf46eb9ae770372adc023b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:00.661392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f782d61baf46eb9ae770372adc023b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:00.662608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: WARNING py.warnings [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:39:00.662608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: warnings.warn(msg) Aug 01 16:39:00.662608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: Aug 01 16:39:00.679644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:00.681304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:00.681449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:00.714930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 848/1643] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:00.878426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:00.884443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f782d61baf46eb9ae770372adc023b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:00.885705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f782d61baf46eb9ae770372adc023b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:00.918553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f782d61baf46eb9ae770372adc023b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:00.936600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46b464-6e0f-4408-9a9c-dbac8e0c9ada service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:00.947453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f782d61baf46eb9ae770372adc023b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:00.948290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-4201e5d8-1254-4e97-98b0-f9fbf4a00b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f782d61baf46eb9ae770372adc023b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:00.969029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:00.969029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:00.969029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:00.969029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] PATH_INFO: `/v3/users/7ebe8558580445f1806be8be7398e2c7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:00.971475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ebe8558580445f1806be8be7398e2c7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:00.975895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:00.990292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:01.010787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 796/1644] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:00 2022] DELETE /identity/v3/users/d7f782d61baf46eb9ae770372adc023b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:01.014274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:01.015847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:01.019316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:01.104961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:01.109629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:01.164621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ebe8558580445f1806be8be7398e2c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:01.165418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ebe8558580445f1806be8be7398e2c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:01.186697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ebe8558580445f1806be8be7398e2c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:01.211490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ebe8558580445f1806be8be7398e2c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:01.230483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2cd7a502-5b2d-46b0-aa1e-7591e5319075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ebe8558580445f1806be8be7398e2c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:01.245221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 849/1645] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:00 2022] DELETE /identity/v3/users/7ebe8558580445f1806be8be7398e2c7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:39:01.250412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:01.250729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:01.251013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:01.296151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 797/1646] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:01.302542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:01.305094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:01.351621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:01.354273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:01.356933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:01.496388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 850/1647] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:01.524815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0350f14-37c0-4276-a337-bc9e65a588f9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:01.551556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:01.555320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:01.690110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:01.694293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:01.695704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:01.697248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:01.726699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:01.728448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:01.771327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 798/1648] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:01.807312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e399e-5a72-42ba-a6b2-288d972f387b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:01.811530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5db2e-d521-4126-b064-a5a7d79b3211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:01.812339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5db2e-d521-4126-b064-a5a7d79b3211 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:01.812696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5db2e-d521-4126-b064-a5a7d79b3211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:01.813031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5db2e-d521-4126-b064-a5a7d79b3211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:01.852928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5db2e-d521-4126-b064-a5a7d79b3211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:01.856906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5db2e-d521-4126-b064-a5a7d79b3211 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:01.881815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 851/1649] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:02.564481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:02.565989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:02.607792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbd2b3-b446-4099-9a47-44c56ea85946 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:02.629307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:02.631658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:02.633642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:02.634476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] PATH_INFO: `/v3/users/7b2dc38f3afd466c8660752a24ad02dd` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:02.637513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2dc38f3afd466c8660752a24ad02dd)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:02.643879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:02.807411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2dc38f3afd466c8660752a24ad02dd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:02.810296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2dc38f3afd466c8660752a24ad02dd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:02.826705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:02.846158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2dc38f3afd466c8660752a24ad02dd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:02.868277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2dc38f3afd466c8660752a24ad02dd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:02.870909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2dc38f3afd466c8660752a24ad02dd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:02.980501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 799/1650] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:02 2022] DELETE /identity/v3/users/7b2dc38f3afd466c8660752a24ad02dd => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:02.986067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:02.986914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:02.987395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:03.025925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:03.029370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:03.198872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 852/1651] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:03.218171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:03.222205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:03.278204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:03.280066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:03.336579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9283b3-adf2-43c3-8ce8-08c9dab55941 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:03.356048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:03.361840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497224-da7d-4340-95cf-d8796f5283e1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:03.366519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6dd07-7a4e-420e-8720-c66d6faf7a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:03.367519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6dd07-7a4e-420e-8720-c66d6faf7a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:03.367904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6dd07-7a4e-420e-8720-c66d6faf7a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:03.369149 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:03.369616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6dd07-7a4e-420e-8720-c66d6faf7a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:03.369997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:03.370607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] PATH_INFO: `/v3/projects/51b7e06f5b96485e8449852d67908bc1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:03.372878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b7e06f5b96485e8449852d67908bc1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:03.374256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: WARNING py.warnings [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:39:03.374256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: warnings.warn(msg) Aug 01 16:39:03.374256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: Aug 01 16:39:03.375988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:03.394425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6dd07-7a4e-420e-8720-c66d6faf7a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:03.396002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6dd07-7a4e-420e-8720-c66d6faf7a38 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:03.423156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 800/1652] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:03.553951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b7e06f5b96485e8449852d67908bc1', 'request_id': 'req-b95d64eb-6a5e-4c49-b881-59673b8b418a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:03.574548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b7e06f5b96485e8449852d67908bc1', 'request_id': 'req-b95d64eb-6a5e-4c49-b881-59673b8b418a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:03.602785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b7e06f5b96485e8449852d67908bc1', 'request_id': 'req-b95d64eb-6a5e-4c49-b881-59673b8b418a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:03.643398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] The token cache is being invalidate because project 51b7e06f5b96485e8449852d67908bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:03.644016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:03.645291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b95d64eb-6a5e-4c49-b881-59673b8b418a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:03.649837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 853/1653] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:03 2022] DELETE /identity/v3/projects/51b7e06f5b96485e8449852d67908bc1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:03.655979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:03.657259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:03.657954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:03.743431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:03.746120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:03.953312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 801/1654] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:03.967883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:03.968682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:03.969208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:03.978259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:03.996880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:04.002845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:04.031314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 854/1655] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:04.037697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.052834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb812d-9e82-4d29-bdfb-8850c61e170e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.060454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.061465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.061931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.062384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.063338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:04.063946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.066478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.091054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 802/1656] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:04.097126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.113468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d5437-9459-45e1-b41b-9b72e75441a0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.122535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.123276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.123611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.124053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.125071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.127983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.137697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.140672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:04.172891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706d212-0b32-465f-92fc-6f2987a33233 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.184940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.185646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.186045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.186338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] PATH_INFO: `/v3/projects/11fb0fc60eac4bf3bbed71ccd34fb778` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.188527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fb0fc60eac4bf3bbed71ccd34fb778)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.190692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.206283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 855/1657] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:04 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:39:04.212286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.313183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fb0fc60eac4bf3bbed71ccd34fb778', 'request_id': 'req-dab84650-1ca7-444e-8ff5-23188ad6e3ed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.323204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:04.327590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fb0fc60eac4bf3bbed71ccd34fb778', 'request_id': 'req-dab84650-1ca7-444e-8ff5-23188ad6e3ed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.352353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fb0fc60eac4bf3bbed71ccd34fb778', 'request_id': 'req-dab84650-1ca7-444e-8ff5-23188ad6e3ed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.362218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70fb6-5c42-46ee-bf9e-1aaf9f7e98f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.375606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] The token cache is being invalidate because project 11fb0fc60eac4bf3bbed71ccd34fb778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:04.375975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.378281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.380228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 803/1658] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:04 2022] DELETE /identity/v3/projects/11fb0fc60eac4bf3bbed71ccd34fb778 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:04.386341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.388362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:04.473190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab84650-1ca7-444e-8ff5-23188ad6e3ed admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.480207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.481062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.481403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.481875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] PATH_INFO: `/v3/users/29aa2e8e6adc41ac804ab41931250080` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.483263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] RBAC: Authorizing `identity:delete_user(user_id=29aa2e8e6adc41ac804ab41931250080)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.485453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.500646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.501303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.501583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.501859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.502625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.504735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.549417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 856/1659] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:04 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:39:04.559229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.564819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:04.612729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c96ea-9f4a-432c-9244-df48306161df admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.625930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29aa2e8e6adc41ac804ab41931250080'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.627734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29aa2e8e6adc41ac804ab41931250080'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.630255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.630933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.631373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.632003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.633310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.638110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.681372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 857/1660] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:04.682840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.705109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d82c-30d0-4c5d-b7fc-39ede7a166d4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.711311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29aa2e8e6adc41ac804ab41931250080'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.720053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.720812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.721205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.721561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.722379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.724235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.731704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29aa2e8e6adc41ac804ab41931250080'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.732985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-31979769-8ee4-4e20-b75f-fb926484021b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29aa2e8e6adc41ac804ab41931250080'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:04.741459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 858/1661] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:04.742734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:04.744920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:04.759850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01860f-707e-4fda-aca5-b2cc7c70bea4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:04.766394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:04.767202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.767554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.767951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] PATH_INFO: `/v3/projects/053023b614f34fa4a7fc6153fbdc986a/users/1b10ca42f1bb4be6a74603fa214831e5/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.780406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 804/1662] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:04 2022] DELETE /identity/v3/users/29aa2e8e6adc41ac804ab41931250080 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:39:04.791756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=053023b614f34fa4a7fc6153fbdc986a, user_id=1b10ca42f1bb4be6a74603fa214831e5, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:04.794665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:04.815160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 859/1663] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:04 2022] PUT /identity/v3/projects/053023b614f34fa4a7fc6153fbdc986a/users/1b10ca42f1bb4be6a74603fa214831e5/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:04.832929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:04.833496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:04.834039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:04.907175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] MFA Rules not processed for user `1b10ca42f1bb4be6a74603fa214831e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:04.909317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:05.056872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:05.123886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 805/1664] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:05.132699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:05.208572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:05.277141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f68e27-7aa9-4334-9ff9-e5288496f5cb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:05.283520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:05.284556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:05.285032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:05.285556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:05.286665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:05.289567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:05.339163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 860/1665] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:05 2022] POST /identity/v3/projects => generated 373 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:05.339367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:05.356733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bd361-da74-44d2-a797-f79a79b4715f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:05.448151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:05.449228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:05.449761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:05.450232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:05.451480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:05.462769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:05.483338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:05.485757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:05.527690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923543a9-b071-4c28-b818-81f647b9cef4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:05.534865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 806/1666] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:05 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:05.543811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:05.551899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:05.553531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:05.555221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:05.556375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] PATH_INFO: `/v3/projects/d825d47bf1a64c19b117d7a62bb69fc0` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:05.560409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d825d47bf1a64c19b117d7a62bb69fc0)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:05.567903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:05.605105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972b999-7c33-46e5-8a24-0415f56d55a8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:05.636180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:05.637319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:05.637780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:05.642940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:05.644393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:05.660652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:05.709622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 807/1667] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:05.711270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:05.724171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01fc5a-4502-4189-a38c-bd9a636b6b5c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:05.734964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd825d47bf1a64c19b117d7a62bb69fc0', 'request_id': 'req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:05.759186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd825d47bf1a64c19b117d7a62bb69fc0', 'request_id': 'req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:05.796794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd825d47bf1a64c19b117d7a62bb69fc0', 'request_id': 'req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:05.833228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] The token cache is being invalidate because project d825d47bf1a64c19b117d7a62bb69fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:05.833557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:05.835411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:05.838433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 861/1668] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:05 2022] DELETE /identity/v3/projects/d825d47bf1a64c19b117d7a62bb69fc0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:05.843844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:05.848725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:05.917152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:05.918759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:05.919410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:05.921960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] PATH_INFO: `/v3/projects/ad42d2c255ba4cfc978603747f541904/users/b0504a180e274d68961e3a1e81e21c79/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:05.979527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad42d2c255ba4cfc978603747f541904, user_id=b0504a180e274d68961e3a1e81e21c79, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:05.986529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:06.010135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772de0e-b5e4-4f3e-9e8b-e0eca709b3a8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:06.028442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 808/1669] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:05 2022] PUT /identity/v3/projects/ad42d2c255ba4cfc978603747f541904/users/b0504a180e274d68961e3a1e81e21c79/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:06.035252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:06.039833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:06.040785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:06.042469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.043751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.045340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] PATH_INFO: `/v3/users/0d338786949c4ffdbb92863c8c863643` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.050222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d338786949c4ffdbb92863c8c863643)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:06.059621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:06.193524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:06.205272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:06.214748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d5c29-9fc7-4d37-a269-9357dc13fa80 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:06.230993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d338786949c4ffdbb92863c8c863643'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.234455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d338786949c4ffdbb92863c8c863643'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.239476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:06.244414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.244593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.244923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.247155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:06.249485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:06.259333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d338786949c4ffdbb92863c8c863643'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.284328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 809/1670] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:06.285291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d338786949c4ffdbb92863c8c863643'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.286584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:06.305974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa8f78a2-1ae0-47e5-b430-0ab2ec82843c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d338786949c4ffdbb92863c8c863643'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.314819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd962f08-0989-4a5a-92d9-e41047554c0a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:06.323638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 862/1671] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:05 2022] DELETE /identity/v3/users/0d338786949c4ffdbb92863c8c863643 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:06.325850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:06.326737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.327023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.327321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] PATH_INFO: `/v3/projects/ad42d2c255ba4cfc978603747f541904/users/b0504a180e274d68961e3a1e81e21c79/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.332313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad42d2c255ba4cfc978603747f541904, user_id=b0504a180e274d68961e3a1e81e21c79, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:06.333231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:06.348875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 810/1672] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:06 2022] PUT /identity/v3/projects/ad42d2c255ba4cfc978603747f541904/users/b0504a180e274d68961e3a1e81e21c79/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:06.371820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.372320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.372804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.426817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:06.429205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:06.449982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] MFA Rules not processed for user `b0504a180e274d68961e3a1e81e21c79`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:06.452542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:06.557188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 863/1673] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:06.570506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:06.573136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:06.577900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3debd4-ded2-45bb-86c2-2d3a4fc06725 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:06.594491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:06.595269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.595656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.596003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] PATH_INFO: `/v3/projects/40d6385d2fd94e4e99c3e79385c23013` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.597546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d6385d2fd94e4e99c3e79385c23013)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:06.599612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:06.658497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f7083-e570-4964-ac50-dd4910bcdbfb None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:06.668317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d1af2-2cb9-4efb-ac5c-506768a4c383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:06.668578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d1af2-2cb9-4efb-ac5c-506768a4c383 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.668662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d1af2-2cb9-4efb-ac5c-506768a4c383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.673437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d1af2-2cb9-4efb-ac5c-506768a4c383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.697588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d1af2-2cb9-4efb-ac5c-506768a4c383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:06.701702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d1af2-2cb9-4efb-ac5c-506768a4c383 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:06.727809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 864/1674] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:06.793578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d6385d2fd94e4e99c3e79385c23013', 'request_id': 'req-c0a96b28-12b7-44f8-926a-df25ef73e4c7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.823040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d6385d2fd94e4e99c3e79385c23013', 'request_id': 'req-c0a96b28-12b7-44f8-926a-df25ef73e4c7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.858625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d6385d2fd94e4e99c3e79385c23013', 'request_id': 'req-c0a96b28-12b7-44f8-926a-df25ef73e4c7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.880065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] The token cache is being invalidate because project 40d6385d2fd94e4e99c3e79385c23013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:06.880471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.887072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:06.908426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 811/1675] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:06 2022] DELETE /identity/v3/projects/40d6385d2fd94e4e99c3e79385c23013 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:06.914122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf8663-7b37-4207-9fe8-3cf86cecef22 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:06.914393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf8663-7b37-4207-9fe8-3cf86cecef22 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:06.914678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf8663-7b37-4207-9fe8-3cf86cecef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:06.928187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:06.930548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:06.988808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-46bf8663-7b37-4207-9fe8-3cf86cecef22 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:06.990165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-46bf8663-7b37-4207-9fe8-3cf86cecef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:07.069284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96b28-12b7-44f8-926a-df25ef73e4c7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:07.083195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:07.085588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:07.086332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:07.086831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:07.090801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:07.093991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:07.144073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:07.200072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 865/1676] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:07.220626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:07.220626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:07.220626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:07.286366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:07.290779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:07.312793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:07.314960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:07.350680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 812/1677] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:07.390113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 866/1678] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:07.397140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:07.449878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2d578-cdb6-4adc-a806-00c7eca4584b service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:07.480249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:07.481849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:07.482349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:07.482820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:07.484297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:07.484830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:07.488314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:07.557194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 813/1679] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:07.562759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:07.593632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc39b0-8c89-4eb2-9dcf-87e5268b33eb None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:07.640478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:07.641621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:07.642116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:07.642603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:07.661221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:07.678104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:07.814261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 867/1680] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:07 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:07.814694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:07.844515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9cea4-6c51-483f-a40e-0da6a68e20a0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:07.967981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:07.968800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:07.969186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:07.969561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:07.970518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:07.972650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:08.018309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 814/1681] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:07 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:08.031730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:08.096123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1160a7-8f67-4bbc-a879-daddcd5f7cdb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:08.115179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:08.119068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:08.119362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:08.119645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:08.120441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:08.122184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:08.164184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 868/1682] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:08.167394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:08.181323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f19678-4289-443b-bb2c-5e849c5bf218 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:08.201924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982bb85-b332-409c-890d-b1630da93068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:08.203301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982bb85-b332-409c-890d-b1630da93068 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:08.203880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982bb85-b332-409c-890d-b1630da93068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:08.204391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982bb85-b332-409c-890d-b1630da93068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:08.205347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982bb85-b332-409c-890d-b1630da93068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:08.207582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982bb85-b332-409c-890d-b1630da93068 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:08.227627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 815/1683] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:08.232455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:08.285226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce724-3640-412a-b0b0-d1bbdccc193e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:08.307537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:08.308679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:08.309222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:08.309930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] PATH_INFO: `/v3/projects/d498c548ae3047388428322e791b29aa/users/4b218646e4384e82a7250b5863f6b26d/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:08.345086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d498c548ae3047388428322e791b29aa, user_id=4b218646e4384e82a7250b5863f6b26d, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:08.365565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:08.415581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 869/1684] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:08 2022] PUT /identity/v3/projects/d498c548ae3047388428322e791b29aa/users/4b218646e4384e82a7250b5863f6b26d/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:08.427125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:08.427125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:08.427125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:08.471899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] MFA Rules not processed for user `4b218646e4384e82a7250b5863f6b26d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:08.473010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:08.706984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 816/1685] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:08.708384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:08.997875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bebc5-4da5-4e6c-8494-3e7a3db951aa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:09.023709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:09.024886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.025378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.025893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:09.026903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:09.031170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:09.093108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:09.094881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:09.106976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 870/1686] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:08 2022] POST /identity/v3/projects => generated 383 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:09.112337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:09.124273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e01d-4066-4bb6-a08a-715301271da3 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:09.163998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:09.279883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b832-86c6-49e8-95dc-5acb0bedd4bf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:09.285870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:09.287145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.287957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.288330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:09.289178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:09.292404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:09.357553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:09.387031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:09.387031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.387031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.390193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] PATH_INFO: `/v3/projects/8d3a811a1b0f43999fbfa1c256e95da6` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:09.394214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 871/1687] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:09 2022] POST /identity/v3/users => generated 298 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:09.394575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3a811a1b0f43999fbfa1c256e95da6)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:09.407498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:09.419855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:09.439216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335160d1-3472-45ff-b55b-5d1f8cc7ddd4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:09.457411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:09.458205 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.459789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.460894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:09.462718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:09.468151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:09.498972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 872/1688] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:09.504879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:09.559872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ce8a0-d2e4-4f41-8915-f88530b59912 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:09.608034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3a811a1b0f43999fbfa1c256e95da6', 'request_id': 'req-c4beccfd-e4e6-4159-a00f-73638223fa95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:09.643184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3a811a1b0f43999fbfa1c256e95da6', 'request_id': 'req-c4beccfd-e4e6-4159-a00f-73638223fa95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:09.676091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3a811a1b0f43999fbfa1c256e95da6', 'request_id': 'req-c4beccfd-e4e6-4159-a00f-73638223fa95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:09.756944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] The token cache is being invalidate because project 8d3a811a1b0f43999fbfa1c256e95da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:09.760388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:09.765427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c4beccfd-e4e6-4159-a00f-73638223fa95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:09.779535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 817/1689] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:09 2022] DELETE /identity/v3/projects/8d3a811a1b0f43999fbfa1c256e95da6 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:09.785433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c1db-4672-476c-893e-63b5a62407b7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.785777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c1db-4672-476c-893e-63b5a62407b7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.787493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48c1db-4672-476c-893e-63b5a62407b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:09.870034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1c48c1db-4672-476c-893e-63b5a62407b7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:09.879518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1c48c1db-4672-476c-893e-63b5a62407b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:09.903430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:09.921013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.921390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.921819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] PATH_INFO: `/v3/projects/6c31f8f53b0a4ae0ab1d6e8166920a70/users/22f41c5775f64dfdbf7387c4bfcc4013/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:09.928009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 818/1690] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:09.938690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:09.939440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:09.939864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.025865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c31f8f53b0a4ae0ab1d6e8166920a70, user_id=22f41c5775f64dfdbf7387c4bfcc4013, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:10.038556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:10.048296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:10.049595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:10.084677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 819/1691] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:10.090000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:10.111515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96158151-4e02-4e1b-bfa8-c64e1be970b5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:10.142756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 873/1692] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:09 2022] PUT /identity/v3/projects/6c31f8f53b0a4ae0ab1d6e8166920a70/users/22f41c5775f64dfdbf7387c4bfcc4013/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:10.151149 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:10.155577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:10.346262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:10.346967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.347290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.347566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.348974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:10.349283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:10.351091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:10.381698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 820/1693] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:10.382620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:10.417469 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171993-6dc2-4022-beb4-e7c9381d57a8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:10.438166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982542dc-f892-4b9c-bc88-99397ed02252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:10.438792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982542dc-f892-4b9c-bc88-99397ed02252 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.439064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982542dc-f892-4b9c-bc88-99397ed02252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.439331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982542dc-f892-4b9c-bc88-99397ed02252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.440066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982542dc-f892-4b9c-bc88-99397ed02252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:10.445764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982542dc-f892-4b9c-bc88-99397ed02252 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:10.461716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae404a0-2d98-4cfb-93fd-007e328881e4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:10.500995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:10.501978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.502435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.502882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.504219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:10.510024 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:10.536094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 821/1694] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:10 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:10.540802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.541092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.541363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.559535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 874/1695] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:10.560669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:10.564746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:10.583855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb3632-1ec6-411f-8059-4bc012e9b235 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:10.620263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] MFA Rules not processed for user `7a310f36f74e4eaabd9f32220feb274a`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:10.621615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:10.721996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:10.722989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.723410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.723928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.724883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:10.727393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:10.743363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:10.786680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 875/1696] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:10 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:10.791796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:10.807556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518a500-84d1-4aef-b75a-b9e406116268 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:10.823965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3f199-8b2e-4730-a3ff-5665c3385403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:10.824761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3f199-8b2e-4730-a3ff-5665c3385403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.825046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3f199-8b2e-4730-a3ff-5665c3385403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.825308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3f199-8b2e-4730-a3ff-5665c3385403 admin admin] PATH_INFO: `/v3/projects/6c31f8f53b0a4ae0ab1d6e8166920a70/users/22f41c5775f64dfdbf7387c4bfcc4013/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.828978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 822/1697] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:10.829955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:10.831751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3f199-8b2e-4730-a3ff-5665c3385403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c31f8f53b0a4ae0ab1d6e8166920a70, user_id=22f41c5775f64dfdbf7387c4bfcc4013, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:10.835366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3f199-8b2e-4730-a3ff-5665c3385403 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:10.856851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c8cdd-0aa7-4854-ac4d-276b5f2acaf9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:10.859051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 876/1698] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:10 2022] PUT /identity/v3/projects/6c31f8f53b0a4ae0ab1d6e8166920a70/users/22f41c5775f64dfdbf7387c4bfcc4013/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:10.874395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:10.874962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:10.875521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:10.984569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] MFA Rules not processed for user `7a310f36f74e4eaabd9f32220feb274a`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:10.986107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:11.024996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a8899-87f6-44b6-8768-8ff98a02228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:11.026063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8899-87f6-44b6-8768-8ff98a02228d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.026508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8899-87f6-44b6-8768-8ff98a02228d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.026972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8899-87f6-44b6-8768-8ff98a02228d admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.034368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8899-87f6-44b6-8768-8ff98a02228d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:11.038944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8899-87f6-44b6-8768-8ff98a02228d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:11.098341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 823/1699] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:11.098341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.101537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.102117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.161706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 877/1700] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:11.162984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:11.183027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] MFA Rules not processed for user `22f41c5775f64dfdbf7387c4bfcc4013`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:11.185472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:11.190782 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3dcba-0bd7-49db-83d9-f7629615a3ad None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:11.203809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:11.203809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.203809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.203809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.204375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:11.206420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:11.234442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 878/1701] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:11.234958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:11.245532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:11.454934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 824/1702] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:11.454934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:11.492060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32136410-e428-4255-acd2-13244a10d7d0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:11.516911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:11.519008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.520836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.521442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] PATH_INFO: `/v3/projects/8659ffebcce441de8f9611f9c00d6dfb/users/ff6e392c579b45e3aea24df86a555e59/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.550664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18920fe-e673-4ad1-b010-a6a5946af309 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:11.566615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:11.573884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b43ee2-7004-4ce3-8083-c66fd2d993b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:11.574733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] RBAC: Authorizing `identity:create_grant(project_id=8659ffebcce441de8f9611f9c00d6dfb, user_id=ff6e392c579b45e3aea24df86a555e59, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:11.575696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b43ee2-7004-4ce3-8083-c66fd2d993b7 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.578507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b43ee2-7004-4ce3-8083-c66fd2d993b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.579473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b43ee2-7004-4ce3-8083-c66fd2d993b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.580055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:11.606890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b43ee2-7004-4ce3-8083-c66fd2d993b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:11.611378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b43ee2-7004-4ce3-8083-c66fd2d993b7 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:11.613917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 825/1703] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:11 2022] PUT /identity/v3/projects/8659ffebcce441de8f9611f9c00d6dfb/users/ff6e392c579b45e3aea24df86a555e59/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:11.614858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:11.618936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:11.639732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 879/1704] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:11.641081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.641825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.642270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.747034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] MFA Rules not processed for user `ff6e392c579b45e3aea24df86a555e59`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:11.749620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:11.807235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ca5f-ace0-4ce5-b142-d2fd7a12c2db admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:11.813744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:11.814464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:11.814743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:11.815062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:11.897101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:11.941605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:11.942495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:11.959163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 826/1705] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:11.974345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 880/1706] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:11.983440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:12.022449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:12.229707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a24201-ff55-4cb4-99a4-8c057b25f20d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:12.237904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfb010-9205-42fd-bee0-2e525c245749 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:12.246485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:12.246847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:12.247356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.247695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.248035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.248249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.248594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.249001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] PATH_INFO: `/v3/users/4b218646e4384e82a7250b5863f6b26d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.249119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:12.250940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b218646e4384e82a7250b5863f6b26d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:12.253176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:12.260351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:12.340991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 827/1707] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:11 2022] POST /identity/v3/projects => generated 363 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:12.343266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:12.375112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b218646e4384e82a7250b5863f6b26d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:12.375849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b218646e4384e82a7250b5863f6b26d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:12.407402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b218646e4384e82a7250b5863f6b26d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:12.433241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b218646e4384e82a7250b5863f6b26d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:12.456405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6a894f8c-9fb6-45b1-9cd2-d92a4222e148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b218646e4384e82a7250b5863f6b26d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:12.473561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 881/1708] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:12 2022] DELETE /identity/v3/users/4b218646e4384e82a7250b5863f6b26d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:12.479655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860383df-12d8-4d5d-be85-c1c454d8c2aa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.480000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860383df-12d8-4d5d-be85-c1c454d8c2aa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.480340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860383df-12d8-4d5d-be85-c1c454d8c2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.489279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:12.559397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-860383df-12d8-4d5d-be85-c1c454d8c2aa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:12.565881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-860383df-12d8-4d5d-be85-c1c454d8c2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:12.649600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fd1b6-755f-4b37-8f46-45bd1f29fff8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:12.692842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 882/1709] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:12.695323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2670e-a8e2-4429-a8d7-d94cc6fbe70c None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.695582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2670e-a8e2-4429-a8d7-d94cc6fbe70c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.695850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2670e-a8e2-4429-a8d7-d94cc6fbe70c None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.698416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 883/1710] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 16:39:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:12.704841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2670e-a8e2-4429-a8d7-d94cc6fbe70c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:12.707287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dfe2670e-a8e2-4429-a8d7-d94cc6fbe70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:12.716292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:12.800631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2670e-a8e2-4429-a8d7-d94cc6fbe70c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:12.806104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41e2f3-0c15-44cd-9407-2bfdf4c6c7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:12.806873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e2f3-0c15-44cd-9407-2bfdf4c6c7cf service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.807861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e2f3-0c15-44cd-9407-2bfdf4c6c7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.807861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e2f3-0c15-44cd-9407-2bfdf4c6c7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.822420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41e2f3-0c15-44cd-9407-2bfdf4c6c7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:12.824467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41e2f3-0c15-44cd-9407-2bfdf4c6c7cf service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:12.842819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 884/1711] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:12.856441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:12.857107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.857413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.857684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.858629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:12.859482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:12.859482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:12.863037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:12.869028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:12.948180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 828/1712] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:12 2022] POST /identity/v3/users => generated 288 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:12.952329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:12.963456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:12.997364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] MFA Rules not processed for user `d85bf2b530ce423d8f1c010b5044dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:12.999210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:13.025009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9225c1-1e66-4dba-92c5-77544dbd13f6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:13.025366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 885/1713] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:13.040327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:13.043423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:13.044451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.045106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.048354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.051215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:13.055352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d2acd-fc5c-4e4c-be0b-0d4e3f1755f6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:13.065887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:13.072639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:13.072639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.073023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.073158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.095706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:13.097325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:13.101906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 829/1714] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:13.108591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:13.112586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 886/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:13.132503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7ac16-9c48-4492-9c88-c4c6e7d5a6d8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:13.141349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:13.142131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.142474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.142850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] PATH_INFO: `/v3/projects/50af6ab58023496ba771d4afaef3184a/users/9f397e73246a445cad29477c656aa929/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.214685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=50af6ab58023496ba771d4afaef3184a, user_id=9f397e73246a445cad29477c656aa929, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:13.237333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:13.279482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 830/1716] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:13 2022] PUT /identity/v3/projects/50af6ab58023496ba771d4afaef3184a/users/9f397e73246a445cad29477c656aa929/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:13.283789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:13.307689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878901-a76b-4dbe-8c14-3821c1864d85 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:13.496681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a297678-cfcc-4edb-989e-82993d68e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:13.497400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a297678-cfcc-4edb-989e-82993d68e273 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.497683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a297678-cfcc-4edb-989e-82993d68e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.497960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a297678-cfcc-4edb-989e-82993d68e273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.498682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a297678-cfcc-4edb-989e-82993d68e273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:13.504328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a297678-cfcc-4edb-989e-82993d68e273 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:13.538091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:13.545001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 887/1717] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:13.551636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cb679-d3fe-4cab-93e4-519fff121eee admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:13.559301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:13.560060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.560384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.560809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] PATH_INFO: `/v3/projects/50af6ab58023496ba771d4afaef3184a/users/9f397e73246a445cad29477c656aa929/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.565218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50af6ab58023496ba771d4afaef3184a, user_id=9f397e73246a445cad29477c656aa929, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:13.568556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:13.585931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 831/1718] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:13 2022] PUT /identity/v3/projects/50af6ab58023496ba771d4afaef3184a/users/9f397e73246a445cad29477c656aa929/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:39:13.596049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fbf52-06f8-4b52-bf5d-591467894e3e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.596664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fbf52-06f8-4b52-bf5d-591467894e3e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.596875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fbf52-06f8-4b52-bf5d-591467894e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.646875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a34fbf52-06f8-4b52-bf5d-591467894e3e None None] MFA Rules not processed for user `9f397e73246a445cad29477c656aa929`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:13.649132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a34fbf52-06f8-4b52-bf5d-591467894e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:13.786722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:13.813371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 888/1719] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:13.821713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:13.824282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:13.985146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fdfe-f5ad-4dbb-beae-8580b9b044be None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.985608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fdfe-f5ad-4dbb-beae-8580b9b044be None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.986890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fdfe-f5ad-4dbb-beae-8580b9b044be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:13.993823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004163e-537a-460c-9d27-2196c70a9da3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:13.997982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:13.998723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:13.999066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:13.999399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:14.013850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:14.023852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:14.054315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 832/1720] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:14.122823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4664fdfe-f5ad-4dbb-beae-8580b9b044be None None] MFA Rules not processed for user `4b0aed0278344119aae6906f4f52b1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:14.125849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4664fdfe-f5ad-4dbb-beae-8580b9b044be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:14.407513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 889/1721] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:14.431331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:14.433773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:14.469765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da323d6b-8ade-4b48-8d0a-5a37ebce3afd None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:14.470137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da323d6b-8ade-4b48-8d0a-5a37ebce3afd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:14.470490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da323d6b-8ade-4b48-8d0a-5a37ebce3afd None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:14.472439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 890/1722] 10.222.0.45 () {54 vars in 961 bytes} [Mon Aug 1 16:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:14.478704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:14.479570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:14.479943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:14.512331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:14.552100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:14.562309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:14.856796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa07729-c561-4e6f-8597-a023c84d7e09 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:14.894347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:14.895647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:14.897586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:14.898186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:14.917726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 891/1723] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Aug 1 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:14.918650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:14.924308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:14.925859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:14.930667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:14.972490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 833/1724] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:14.973750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:15.015207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73222355-2a07-4283-b055-84a45e856f52 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:15.037361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:15.038251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:15.039462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:15.039953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:15.048031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:15.086237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:15.088521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:15.123121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 834/1725] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:15.201810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f69e84-ea21-4cd8-9415-fb29f2565bc2 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:15.211415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:15.212831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:15.232069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:15.233311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] PATH_INFO: `/v3/projects/d498c548ae3047388428322e791b29aa` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:15.235268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d498c548ae3047388428322e791b29aa)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:15.237647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:15.391877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:15.402877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:15.450515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b5aec-e2bc-4056-a5e6-4f3524dd3093 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:15.513136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd498c548ae3047388428322e791b29aa', 'request_id': 'req-2d106185-3761-4966-bacf-93dd949d5ebf'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:15.538819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd498c548ae3047388428322e791b29aa', 'request_id': 'req-2d106185-3761-4966-bacf-93dd949d5ebf'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:15.565780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd498c548ae3047388428322e791b29aa', 'request_id': 'req-2d106185-3761-4966-bacf-93dd949d5ebf'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:15.628517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] The token cache is being invalidate because project d498c548ae3047388428322e791b29aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:15.628868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:15.633888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:15.642638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 892/1726] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:14 2022] DELETE /identity/v3/projects/d498c548ae3047388428322e791b29aa => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:15.647865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:15.655780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:15.772806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:15.773655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:15.773939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:15.774213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] PATH_INFO: `/v3/users/ff6e392c579b45e3aea24df86a555e59` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:15.776682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6e392c579b45e3aea24df86a555e59)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:15.779436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:15.961883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6e392c579b45e3aea24df86a555e59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:15.964678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6e392c579b45e3aea24df86a555e59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.025353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6e392c579b45e3aea24df86a555e59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.055971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6e392c579b45e3aea24df86a555e59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.059498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-881f9a3a-449e-479b-a3aa-72630d2ad80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6e392c579b45e3aea24df86a555e59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.089938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d106185-3761-4966-bacf-93dd949d5ebf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:16.104255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 835/1727] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:15 2022] DELETE /identity/v3/users/ff6e392c579b45e3aea24df86a555e59 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:16.109899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:16.110200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:16.110458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:16.116150 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:16.127320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:16.127320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:16.127961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] PATH_INFO: `/v3/users/22f41c5775f64dfdbf7387c4bfcc4013` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:16.133418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f41c5775f64dfdbf7387c4bfcc4013)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:16.141101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:16.197268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:16.202592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:16.222171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:16.364577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f41c5775f64dfdbf7387c4bfcc4013'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.369778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f41c5775f64dfdbf7387c4bfcc4013'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.402059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f41c5775f64dfdbf7387c4bfcc4013'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.424629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 836/1728] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:16.435004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f41c5775f64dfdbf7387c4bfcc4013'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.460551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f41c5775f64dfdbf7387c4bfcc4013'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:16.477174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 893/1729] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:15 2022] DELETE /identity/v3/users/22f41c5775f64dfdbf7387c4bfcc4013 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:16.533306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:16.534840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:16.806361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcf35d-1efd-4ee5-95df-9f4a9829ecc4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:17.131737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:17.137256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:17.137967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:17.138550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:17.143465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:17.530010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:17.541380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72278ee3-01ad-4759-92d7-15b57dd69d1b service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:17.593516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 837/1730] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1061 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:19.585228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:19.587409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:19.646818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744718b9-5fc0-4467-8493-f6a23f8db450 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:19.664309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:19.665807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:19.666446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:19.668169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] PATH_INFO: `/v3/projects/6c31f8f53b0a4ae0ab1d6e8166920a70` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:19.681139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c31f8f53b0a4ae0ab1d6e8166920a70)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:19.685547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:19.686215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:19.686353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:19.686353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:19.762548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:19.839396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:19.840612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:19.842231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:19.906085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c31f8f53b0a4ae0ab1d6e8166920a70', 'request_id': 'req-3e814bb1-b9a4-4102-8769-fce7ccc45434'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:19.929105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c31f8f53b0a4ae0ab1d6e8166920a70', 'request_id': 'req-3e814bb1-b9a4-4102-8769-fce7ccc45434'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:19.961724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c31f8f53b0a4ae0ab1d6e8166920a70', 'request_id': 'req-3e814bb1-b9a4-4102-8769-fce7ccc45434'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:19.998765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] The token cache is being invalidate because project 6c31f8f53b0a4ae0ab1d6e8166920a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:19.999704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:20.006909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3e814bb1-b9a4-4102-8769-fce7ccc45434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:20.015331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 894/1731] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:19 2022] DELETE /identity/v3/projects/6c31f8f53b0a4ae0ab1d6e8166920a70 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:20.017869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2754e6-39da-4556-94a7-74e4c899bae7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:20.028162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2754e6-39da-4556-94a7-74e4c899bae7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:20.028770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2754e6-39da-4556-94a7-74e4c899bae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:20.090313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 838/1732] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:39:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:20.118316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7e2754e6-39da-4556-94a7-74e4c899bae7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:20.120586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7e2754e6-39da-4556-94a7-74e4c899bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:20.508181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:20.517960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:20.591045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 895/1733] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 575 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:20.606887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:20.607395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:20.612182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:20.756624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:20.759239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:20.777127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 896/1734] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:20.784081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:20.809007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8be2e3-c179-4353-ab6e-b72b9e6ad1e6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:20.825547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:20.827742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:20.828297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:20.828987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:20.833201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:20.834719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:20.839518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:20.887930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 897/1735] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:20.894609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:20.939438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40910a2e-8f68-438a-9700-95908aeaa13c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:20.950624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:20.951912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:20.952375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:20.952846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:20.954608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:20.959546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:21.034840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ece30-3709-49de-aa18-28f3425e0dd9 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:21.107962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 898/1736] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:20 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:21.124039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:21.144437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:21.452269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:21.453682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:21.454202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:21.454699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:21.461376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384358d-48a8-4887-b1c8-9dd746acb0be admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:21.491961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:21.493057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:21.493519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:21.493986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] PATH_INFO: `/v3/projects/8659ffebcce441de8f9611f9c00d6dfb` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:21.501725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8659ffebcce441de8f9611f9c00d6dfb)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:21.505268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:21.516904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:21.582046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:21.770249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8659ffebcce441de8f9611f9c00d6dfb', 'request_id': 'req-0ed51a65-4c22-4530-a97d-a92c1a23197d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:21.839363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8659ffebcce441de8f9611f9c00d6dfb', 'request_id': 'req-0ed51a65-4c22-4530-a97d-a92c1a23197d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:21.845471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:21.848909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:21.884440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8659ffebcce441de8f9611f9c00d6dfb', 'request_id': 'req-0ed51a65-4c22-4530-a97d-a92c1a23197d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:21.904348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 839/1737] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:39:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1398 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:39:21.905928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:21.946486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f670c2-e97c-4456-8eca-ee706de2c8b4 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:21.977326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] The token cache is being invalidate because project 8659ffebcce441de8f9611f9c00d6dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:21.977638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:21.984658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:21.992153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 899/1738] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:21 2022] DELETE /identity/v3/projects/8659ffebcce441de8f9611f9c00d6dfb => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:22.002342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:22.006085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:22.145707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:22.146519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:22.146799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:22.147070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:22.147947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:22.149782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:22.211170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 840/1739] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:21 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:22.212237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:22.228157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:22.240783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51a65-4c22-4530-a97d-a92c1a23197d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:22.263374 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:22.264262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:22.264585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:22.265358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] PATH_INFO: `/v3/users/9f397e73246a445cad29477c656aa929` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:22.266567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f397e73246a445cad29477c656aa929)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:22.268463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:22.269208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010acb34-6ae1-407c-9a2c-2d3a42e99465 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:22.305166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:22.305399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:22.305669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:22.305887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:22.306628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:22.308534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:22.356885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 841/1740] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:22.368390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:22.398932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d80a1-67f2-46f2-a485-a836113f6629 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:22.416458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f397e73246a445cad29477c656aa929'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:22.417846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:22.418567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:22.418856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:22.419122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:22.419894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:22.421598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:22.422387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f397e73246a445cad29477c656aa929'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:22.448880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f397e73246a445cad29477c656aa929'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:22.460130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 842/1741] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:22.461108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:22.485791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2434-1e75-46c4-9db2-6b4e76f9e5a7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:22.500018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bea4ed-6d12-4aea-97fc-f48dde5011c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:22.500739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea4ed-6d12-4aea-97fc-f48dde5011c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:22.501022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea4ed-6d12-4aea-97fc-f48dde5011c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:22.501297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bea4ed-6d12-4aea-97fc-f48dde5011c2 admin admin] PATH_INFO: `/v3/projects/2e1cf94d8ded45eda0e036fee69e2717/users/db39bb619698409ba0718c8263a20f57/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:22.550100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f397e73246a445cad29477c656aa929'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:22.568254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bea4ed-6d12-4aea-97fc-f48dde5011c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1cf94d8ded45eda0e036fee69e2717, user_id=db39bb619698409ba0718c8263a20f57, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:22.568254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bea4ed-6d12-4aea-97fc-f48dde5011c2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:22.591412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f397e73246a445cad29477c656aa929'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:22.612694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 843/1742] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:22 2022] PUT /identity/v3/projects/2e1cf94d8ded45eda0e036fee69e2717/users/db39bb619698409ba0718c8263a20f57/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:22.614356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:22.614646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:22.615124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:22.640172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 900/1743] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:22 2022] DELETE /identity/v3/users/9f397e73246a445cad29477c656aa929 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:39:22.691495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] MFA Rules not processed for user `db39bb619698409ba0718c8263a20f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:22.692981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:22.792049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:22.858539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 844/1744] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:22.859427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:22.861477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:22.987726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:23.194808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcea6a-0910-425b-9a79-01c4240b117b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:23.216398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:23.219575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:23.221670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:23.221995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:23.222851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:23.236306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:23.404552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 901/1745] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:22 2022] POST /identity/v3/projects => generated 387 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:23.408070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:23.434922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c45f7c-d006-4c80-8532-83bca6b0b7e6 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:23.696317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:23.696984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:23.697263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:23.697536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:23.698237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:23.711130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:23.796143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 845/1746] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:23 2022] POST /identity/v3/users => generated 300 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:23.800062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:23.818383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a35e9-059c-4b71-8d00-36a7beead3eb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:23.840282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:23.840956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:23.843940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:23.856749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:23.857240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:23.866107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:23.916362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 902/1747] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:23.920898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:23.954777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a0707-004a-47bf-99a9-94dff54fe8ec admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:23.989447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:23.990665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:23.991132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:23.991553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] PATH_INFO: `/v3/projects/3bc4e5decebc43759a9c02d5581fbfc7/users/153d1ec21f1f463885291e4d47b9795f/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:24.075023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc4e5decebc43759a9c02d5581fbfc7, user_id=153d1ec21f1f463885291e4d47b9795f, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:24.082813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:24.138537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 846/1748] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:23 2022] PUT /identity/v3/projects/3bc4e5decebc43759a9c02d5581fbfc7/users/153d1ec21f1f463885291e4d47b9795f/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:24.143301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:24.167466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea168e-2af1-4c56-9f43-e9f6781f8958 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:24.340376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e91d4-eab0-406e-a0c8-6364184809d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:24.344380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e91d4-eab0-406e-a0c8-6364184809d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:24.346579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e91d4-eab0-406e-a0c8-6364184809d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:24.347578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e91d4-eab0-406e-a0c8-6364184809d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:24.354407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e91d4-eab0-406e-a0c8-6364184809d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:24.363502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e91d4-eab0-406e-a0c8-6364184809d7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:24.425275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:24.425828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 903/1749] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:24.442316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a64f5b-26e0-4551-a089-2952b9ac1dfb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:24.481830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:24.485977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:24.486807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:24.487432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] PATH_INFO: `/v3/projects/3bc4e5decebc43759a9c02d5581fbfc7/users/153d1ec21f1f463885291e4d47b9795f/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:24.498194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc4e5decebc43759a9c02d5581fbfc7, user_id=153d1ec21f1f463885291e4d47b9795f, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:24.509520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:24.571031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 847/1750] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:24 2022] PUT /identity/v3/projects/3bc4e5decebc43759a9c02d5581fbfc7/users/153d1ec21f1f463885291e4d47b9795f/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:24.586983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:24.586983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:24.587281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:24.700558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] MFA Rules not processed for user `153d1ec21f1f463885291e4d47b9795f`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:24.704880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:24.931583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 904/1751] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:24.964916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:24.967090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:25.125746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dd6f0-5228-4fa6-91f1-886e8bc8bc88 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:25.131610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:25.133715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:25.139932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:25.140480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:25.163641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:25.167515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:25.191910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 848/1752] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:25.408125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:25.458286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:25.575372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b3f2-18ca-4c53-bda0-79cad90b8e7b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:25.580423 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:25.581112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:25.582452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:25.582759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] PATH_INFO: `/v3/users/db39bb619698409ba0718c8263a20f57` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:25.585738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db39bb619698409ba0718c8263a20f57)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:25.605265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:25.722263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:25.724424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:25.755343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0d428-24c4-4951-8fa3-910cb2eb81fc service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:25.765080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:25.765978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:25.766146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:25.766299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] PATH_INFO: `/v3/projects/50af6ab58023496ba771d4afaef3184a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:25.768622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50af6ab58023496ba771d4afaef3184a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:25.770736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:25.874734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:25.876179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db39bb619698409ba0718c8263a20f57'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.877778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db39bb619698409ba0718c8263a20f57'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.906384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db39bb619698409ba0718c8263a20f57'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.920931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50af6ab58023496ba771d4afaef3184a', 'request_id': 'req-45eb9acd-f3f3-470d-9f06-d08219f25c03'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.946206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50af6ab58023496ba771d4afaef3184a', 'request_id': 'req-45eb9acd-f3f3-470d-9f06-d08219f25c03'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.962976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db39bb619698409ba0718c8263a20f57'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.981549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50af6ab58023496ba771d4afaef3184a', 'request_id': 'req-45eb9acd-f3f3-470d-9f06-d08219f25c03'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:25.995043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-4e37489d-4693-4036-8fbe-3c308abd1be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db39bb619698409ba0718c8263a20f57'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:26.015000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] The token cache is being invalidate because project 50af6ab58023496ba771d4afaef3184a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:26.015289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:26.018919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-45eb9acd-f3f3-470d-9f06-d08219f25c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:26.019799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 905/1753] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:25 2022] DELETE /identity/v3/users/db39bb619698409ba0718c8263a20f57 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:26.026129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4adbc-d1bb-4d78-a50d-197b11ffe514 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:26.026821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 849/1754] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:25 2022] DELETE /identity/v3/projects/50af6ab58023496ba771d4afaef3184a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:26.026945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4adbc-d1bb-4d78-a50d-197b11ffe514 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:26.027521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4adbc-d1bb-4d78-a50d-197b11ffe514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:26.036123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f035-20d6-424c-83e5-35309786ffe7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:26.037112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f035-20d6-424c-83e5-35309786ffe7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:26.037112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92f035-20d6-424c-83e5-35309786ffe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:26.100854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-67a4adbc-d1bb-4d78-a50d-197b11ffe514 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:26.101970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-67a4adbc-d1bb-4d78-a50d-197b11ffe514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:26.110412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ae92f035-20d6-424c-83e5-35309786ffe7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:26.111645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ae92f035-20d6-424c-83e5-35309786ffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:26.464609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 906/1755] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:26.468372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed440f-ca9d-49da-8b9c-7c6ca0f75884 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:26.468372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed440f-ca9d-49da-8b9c-7c6ca0f75884 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:26.468372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed440f-ca9d-49da-8b9c-7c6ca0f75884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:26.549037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 850/1756] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:26.550599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:26.550870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:26.551177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:26.582928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-efed440f-ca9d-49da-8b9c-7c6ca0f75884 None None] MFA Rules not processed for user `d85bf2b530ce423d8f1c010b5044dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:26.588151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-efed440f-ca9d-49da-8b9c-7c6ca0f75884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:26.659986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:26.663604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:26.689566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 851/1757] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:26.702525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:26.707686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:26.765109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:26.841118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 907/1758] 10.222.0.110 () {58 vars in 1016 bytes} [Mon Aug 1 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:26.844708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:26.846553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:26.847923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:26.955640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15a21f-6abd-4dbb-bcac-f426875d492f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:26.975810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:26.980484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:26.980484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:26.980484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:27.015625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] MFA Rules not processed for user `76c1fab92b854cd8937cc80818159191`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:27.017379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:27.021130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:27.029558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:27.070169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 852/1759] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:27.071947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:27.092313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617a79a-f994-4f26-98f2-96dc8856e65b service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:27.102782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:27.104339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:27.104863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:27.105330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:27.107104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:27.107104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:27.110268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:27.175132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 853/1760] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:27.176773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:27.219931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218cb74-f209-4444-9eea-2ee46b6b53a3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:27.236642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:27.237303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:27.237584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:27.237855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:27.248329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:27.249835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:27.280267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 854/1761] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:27.281166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:27.308477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cedb01-9dbb-4613-86bc-d0f1a318e33b service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:27.324803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:27.325977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:27.326328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:27.327259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:27.327998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:27.333791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:27.404020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 855/1763] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:27 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:27.404761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 908/1762] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:27.406663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:27.421609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:27.425889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:27.471000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:27.707326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39415c58-7081-44ea-bbd7-db47bcf17dab admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:27.738246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:27.741489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:27.744067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:27.748587 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:27.752833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:27.758135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4be62-9796-4a74-a42e-9ffe3a9b302d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:27.760520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:27.843233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 856/1764] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:27 2022] POST /identity/v3/users => generated 290 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:27.845953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:27.879412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0918d-dc7c-4807-a938-e6938057853e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:27.903722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:27.904926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:27.905409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:27.905932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:27.907265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:27.915014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:27.983491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 857/1765] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:27.985302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:28.049965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9283-36af-4769-9f1c-0b19397d2505 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:28.089483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:28.090898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:28.092255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:28.097850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:28.099694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:28.102128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:28.112483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5749b49-adb7-4460-90c2-e16325a0b8b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:28.116183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5749b49-adb7-4460-90c2-e16325a0b8b2 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:28.117269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5749b49-adb7-4460-90c2-e16325a0b8b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:28.117813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5749b49-adb7-4460-90c2-e16325a0b8b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:28.153382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 858/1766] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:28.167338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:28.180518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:28.231472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4553f76-2895-401d-b718-f85ed82436ba admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:28.261790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:28.263127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:28.263623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:28.264216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] PATH_INFO: `/v3/projects/0ab903b3452f4fefa703328a1dceb1b8/users/60e9edd5bf324e218c72e81819dff5f3/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:28.318416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab903b3452f4fefa703328a1dceb1b8, user_id=60e9edd5bf324e218c72e81819dff5f3, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:28.326669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:28.367181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 859/1767] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:28 2022] PUT /identity/v3/projects/0ab903b3452f4fefa703328a1dceb1b8/users/60e9edd5bf324e218c72e81819dff5f3/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:28.368303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:28.377814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:28.387737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5749b49-adb7-4460-90c2-e16325a0b8b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:28.390004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5749b49-adb7-4460-90c2-e16325a0b8b2 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:28.425778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 909/1768] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 1004 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:28.426304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:28.426596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:28.426898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:28.518894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] MFA Rules not processed for user `60e9edd5bf324e218c72e81819dff5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:28.522938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:28.579287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75395b3-b957-44b7-83e7-20d0de002014 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:28.600075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:28.601157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:28.601636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:28.602067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] PATH_INFO: `/v3/projects/2e1cf94d8ded45eda0e036fee69e2717` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:28.608814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1cf94d8ded45eda0e036fee69e2717)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:28.611120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:28.813927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1cf94d8ded45eda0e036fee69e2717', 'request_id': 'req-c9134a55-bcab-430f-8fc8-ed86f250f17a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:28.817913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 910/1769] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:28.824093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:28.839976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:28.842640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1cf94d8ded45eda0e036fee69e2717', 'request_id': 'req-c9134a55-bcab-430f-8fc8-ed86f250f17a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:28.845817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca50aa-92ca-4616-bb3b-f0ffbf569603 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:28.877597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1cf94d8ded45eda0e036fee69e2717', 'request_id': 'req-c9134a55-bcab-430f-8fc8-ed86f250f17a'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:28.951789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] The token cache is being invalidate because project 2e1cf94d8ded45eda0e036fee69e2717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:28.952344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:28.955972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c9134a55-bcab-430f-8fc8-ed86f250f17a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:28.965464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 860/1770] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:28 2022] DELETE /identity/v3/projects/2e1cf94d8ded45eda0e036fee69e2717 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:28.968857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:28.969319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:28.969607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:29.022339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:29.022924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:29.023206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:29.023492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:29.026754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:29.043951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:29.058579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] MFA Rules not processed for user `4294d6dff5054da7afcb37b8b28f1a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:29.060043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:29.117992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 911/1771] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:28 2022] POST /identity/v3/projects => generated 381 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:29.119052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:29.128937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:29.275558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:29.288699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 861/1772] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:29.288699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:29.297947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:29.383419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2dc39-8101-480c-a481-d5381a963440 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:29.394028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50344a-00be-47f4-aca0-16a2cfe14ca2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:29.395280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:29.396509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:29.397124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:29.397603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] PATH_INFO: `/v3/users/153d1ec21f1f463885291e4d47b9795f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:29.399195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=153d1ec21f1f463885291e4d47b9795f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:29.415707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80234ea4-40d1-46aa-b020-5792525bbede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:29.416857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80234ea4-40d1-46aa-b020-5792525bbede admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:29.417196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80234ea4-40d1-46aa-b020-5792525bbede admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:29.417488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80234ea4-40d1-46aa-b020-5792525bbede admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:29.418710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80234ea4-40d1-46aa-b020-5792525bbede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:29.419254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:29.431612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80234ea4-40d1-46aa-b020-5792525bbede admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:29.517932 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 862/1773] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:29 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:29.524415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f9e4a-bc1d-471e-a540-0fa810bc19a9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:29.524940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f9e4a-bc1d-471e-a540-0fa810bc19a9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:29.525855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f9e4a-bc1d-471e-a540-0fa810bc19a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:29.594515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-292f9e4a-bc1d-471e-a540-0fa810bc19a9 None None] MFA Rules not processed for user `4294d6dff5054da7afcb37b8b28f1a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:29.597009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-292f9e4a-bc1d-471e-a540-0fa810bc19a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:29.619521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153d1ec21f1f463885291e4d47b9795f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:29.624950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153d1ec21f1f463885291e4d47b9795f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:29.654185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153d1ec21f1f463885291e4d47b9795f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:29.719721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153d1ec21f1f463885291e4d47b9795f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:29.760483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153d1ec21f1f463885291e4d47b9795f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:29.809516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 912/1774] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:29 2022] DELETE /identity/v3/users/153d1ec21f1f463885291e4d47b9795f => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:29.815873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:29.826829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:29.920654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 863/1775] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:29.927875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:29.927875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:29.932604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:30.027433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] MFA Rules not processed for user `4b0aed0278344119aae6906f4f52b1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:30.033465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:30.132822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4922f9-089b-4c1d-925e-0181a00c3bbe admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:30.161819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38664-30db-419c-92eb-c87ca8bc89ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:30.162419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38664-30db-419c-92eb-c87ca8bc89ca admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:30.162720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38664-30db-419c-92eb-c87ca8bc89ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:30.162963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38664-30db-419c-92eb-c87ca8bc89ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:30.163711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a38664-30db-419c-92eb-c87ca8bc89ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:30.171691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a38664-30db-419c-92eb-c87ca8bc89ca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:30.200259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 913/1776] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:30.203558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:30.204501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:30.205282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:30.280013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] MFA Rules not processed for user `1ac7d3acf1204a9f98f9ee14c2b46a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:30.282630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:30.303196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 864/1777] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:30.304912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:30.316402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:30.352055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:30.356391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:30.468039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 914/1778] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:30.483929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:30.508930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdc251-087a-45e8-a6ec-a296cae55f52 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:30.522492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:30.525094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:30.525493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:30.526545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] PATH_INFO: `/v3/projects/2585b4585ef84b07aa160d792cc4d491/users/d563005cae6d444c9068fc8428dcc092/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:30.599520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2585b4585ef84b07aa160d792cc4d491, user_id=d563005cae6d444c9068fc8428dcc092, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:30.615796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:30.642906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c293-78a1-4c24-af5b-a429e912d318 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:30.680048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 915/1779] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:30 2022] PUT /identity/v3/projects/2585b4585ef84b07aa160d792cc4d491/users/d563005cae6d444c9068fc8428dcc092/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:30.681127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:31.092295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:31.095023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:31.099584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:31.100283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:31.102733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a6dbb-d7d4-4bd4-819b-aa0a02b6cf38 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:31.110416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:31.154446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:31.155247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:31.155247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:31.155507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:31.156072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:31.158080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:31.247245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 916/1780] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:31.263034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:31.302978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677ee89-2d86-47d2-9d65-bd1bab51cb9c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:31.328536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9919340-b494-43de-b5ef-262295106fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:31.329871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9919340-b494-43de-b5ef-262295106fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:31.331125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9919340-b494-43de-b5ef-262295106fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:31.333308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9919340-b494-43de-b5ef-262295106fcd admin admin] PATH_INFO: `/v3/projects/2585b4585ef84b07aa160d792cc4d491/users/d563005cae6d444c9068fc8428dcc092/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:31.343869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9919340-b494-43de-b5ef-262295106fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2585b4585ef84b07aa160d792cc4d491, user_id=d563005cae6d444c9068fc8428dcc092, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:31.361079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9919340-b494-43de-b5ef-262295106fcd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:31.400488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 917/1781] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:31 2022] PUT /identity/v3/projects/2585b4585ef84b07aa160d792cc4d491/users/d563005cae6d444c9068fc8428dcc092/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:31.409033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575695b-2eda-4f54-a192-7acfe4833e17 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:31.410791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575695b-2eda-4f54-a192-7acfe4833e17 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:31.411339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575695b-2eda-4f54-a192-7acfe4833e17 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:31.451879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 918/1782] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:39:31 2022] GET /identity => generated 270 bytes in 50 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:31.455828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:31.460725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:31.463822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:31.476577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:31.487751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbdf71-9317-4c6a-a2b6-92224f3c4418 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:31.520233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 865/1783] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 1216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:31.574728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:31.576250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:31.576671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:31.691005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:31.696180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:31.709936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] MFA Rules not processed for user `d563005cae6d444c9068fc8428dcc092`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:31.717162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:31.883101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:31.972337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 866/1784] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:39:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:32.049330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 919/1785] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 596 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:32.097943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:32.100726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:32.230748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:32.235463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:32.292438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:32.351469 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d177f16-d01f-49bf-9296-29867447dd14 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:32.381132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395247a-e676-4e2a-b07f-ed5fe73e315b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:32.382272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395247a-e676-4e2a-b07f-ed5fe73e315b service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:32.382799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395247a-e676-4e2a-b07f-ed5fe73e315b service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:32.383294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395247a-e676-4e2a-b07f-ed5fe73e315b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:32.424781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a42d-91a3-4f31-85b4-c61575fd6a60 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:32.425468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395247a-e676-4e2a-b07f-ed5fe73e315b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:32.428229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395247a-e676-4e2a-b07f-ed5fe73e315b service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:32.436722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:32.438656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:32.439937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:32.440499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] PATH_INFO: `/v3/projects/3bc4e5decebc43759a9c02d5581fbfc7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:32.442093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc4e5decebc43759a9c02d5581fbfc7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:32.446105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:32.499392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 867/1786] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:32.512078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba66d-a161-4221-b7d2-322aa9dcd971 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:32.515649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba66d-a161-4221-b7d2-322aa9dcd971 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:32.517741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba66d-a161-4221-b7d2-322aa9dcd971 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:32.519564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 868/1787] 10.222.0.45 () {54 vars in 912 bytes} [Mon Aug 1 16:39:32 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:32.537512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42b14a-9cf0-4d4b-a7e9-98a096269b70 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:32.538346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42b14a-9cf0-4d4b-a7e9-98a096269b70 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:32.538964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42b14a-9cf0-4d4b-a7e9-98a096269b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:32.565160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc4e5decebc43759a9c02d5581fbfc7', 'request_id': 'req-60000e5e-6728-4763-93e2-78866c768ed5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:32.595031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc4e5decebc43759a9c02d5581fbfc7', 'request_id': 'req-60000e5e-6728-4763-93e2-78866c768ed5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:32.628809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc4e5decebc43759a9c02d5581fbfc7', 'request_id': 'req-60000e5e-6728-4763-93e2-78866c768ed5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:32.652337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2e42b14a-9cf0-4d4b-a7e9-98a096269b70 None None] MFA Rules not processed for user `d85bf2b530ce423d8f1c010b5044dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:32.654756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2e42b14a-9cf0-4d4b-a7e9-98a096269b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:32.676958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] The token cache is being invalidate because project 3bc4e5decebc43759a9c02d5581fbfc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:32.680434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:32.681981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-60000e5e-6728-4763-93e2-78866c768ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:32.693647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 920/1788] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:32 2022] DELETE /identity/v3/projects/3bc4e5decebc43759a9c02d5581fbfc7 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:32.696368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:32.696653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:32.696924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:32.780215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:32.784799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:32.972936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 869/1789] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:39:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:33.192218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 921/1790] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:33.199545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:33.204830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:33.205482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:33.333277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:33.336510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:33.403369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:33.405303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 870/1791] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:33.430685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731efd2b-b5da-4004-b7be-dbe7ace892d8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:33.457925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:33.458786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:33.459226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:33.465817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:33.467750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:33.469043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:33.481256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:33.518049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:33.528150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:33.566176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 922/1792] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:39:33.573876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:33.643355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90cd66-4ce9-46b2-88bd-49104ef0607f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:33.704869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:33.705887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:33.711363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b099d33-2a04-4c86-8e68-9c631c5dce42 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:33.714008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:33.714592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:33.778014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:33.786548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:33.787104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:33.788208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:33.788967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:33.791036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:33.904145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:33.914359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:33.967474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 923/1793] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:33 2022] POST /identity/v3/projects => generated 409 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:33.968876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:33.972478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 871/1794] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:34.027562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:34.139609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:34.142105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:34.360396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cb908-fcff-4cbe-9d22-fe4ef505e5fd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:34.384087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:34.384087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:34.384602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:34.385092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:34.389099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:34.393698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:34.523666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 924/1795] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:33 2022] POST /identity/v3/users => generated 305 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:34.525350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:34.547564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee8da9-ca1a-4b0b-b876-db774713ed04 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:34.619966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e25d0d-e6f0-4dd7-9426-605042c44932 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:34.656474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:34.754435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:34.756437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:34.756966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:34.757675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:34.796030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:34.809348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:34.864379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 925/1796] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:34.868930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:34.899662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ed48a-2aad-400a-8516-be05ae782481 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:34.937410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:34.938362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:34.938899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:34.939938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:34.947800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:34.957887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:35.041553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 926/1797] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:35.046653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:35.113456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433f471-5201-4278-b5c0-2752fe638b9f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:35.150711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:35.150711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:35.150711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:35.151608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] PATH_INFO: `/v3/projects/55db4c94326e4b37bd2643eb792df566/users/82ccfa7249b74b6b8a471d8f964c339b/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:35.214312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd563005cae6d444c9068fc8428dcc092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2585b4585ef84b07aa160d792cc4d491', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:35.218687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:35.221517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:35.223648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:35.226422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:35.230165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:35.255046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:35.271799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55db4c94326e4b37bd2643eb792df566, user_id=82ccfa7249b74b6b8a471d8f964c339b, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:35.330680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:35.444978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 872/1798] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 1306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:35.447791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:35.463488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:35.476178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 927/1799] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:35 2022] PUT /identity/v3/projects/55db4c94326e4b37bd2643eb792df566/users/82ccfa7249b74b6b8a471d8f964c339b/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:35.478001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:35.888837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccd88-5e8e-451d-a3fe-c9976db73259 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:35.920206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec758a-2f76-41f0-b690-fa7e4afdfb6c tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd563005cae6d444c9068fc8428dcc092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2585b4585ef84b07aa160d792cc4d491', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:35.927755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec758a-2f76-41f0-b690-fa7e4afdfb6c tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:35.935177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec758a-2f76-41f0-b690-fa7e4afdfb6c tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:35.939679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec758a-2f76-41f0-b690-fa7e4afdfb6c tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:35.942603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec758a-2f76-41f0-b690-fa7e4afdfb6c tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:35.944947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808e18b-3e9a-4090-bcee-45d8c3b042e2 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:35.950569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec758a-2f76-41f0-b690-fa7e4afdfb6c tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:36.164618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 928/1800] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:35 2022] POST /identity/v3/projects => generated 354 bytes in 687 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:36.176008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:36.176409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:36.177211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:36.213651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:36.214615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:36.215040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:36.216182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:36.218498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:36.286348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] MFA Rules not processed for user `82ccfa7249b74b6b8a471d8f964c339b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:36.288671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:36.361693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:36.429661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:36.438485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:36.440226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:36.501196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 873/1801] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 1055 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:36.502231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:36.562930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bfae7-f804-452c-904c-412860fa2ed9 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:36.600938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 929/1802] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:36.610075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:36.792502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd563005cae6d444c9068fc8428dcc092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2585b4585ef84b07aa160d792cc4d491', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:36.793375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:36.793537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:36.793751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:36.794542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:36.794903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:36.797490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:36.847024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 874/1803] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:36.848180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:36.852717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:36.855151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59856-28c3-43d8-96ee-83422720862c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:36.884621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:36.885724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:36.886186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:36.886665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:36.887613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:36.893484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:36.979542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 930/1804] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:36 2022] POST /identity/v3/projects => generated 409 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:36.983019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:37.179062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90110284-babf-48b0-84a9-cfb9d94951fd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:37.185135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd563005cae6d444c9068fc8428dcc092', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2585b4585ef84b07aa160d792cc4d491', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:37.185917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:37.186268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:37.186630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] PATH_INFO: `/v3/projects/8a1c42d1d52040cab22142f15d53875a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:37.189167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8a1c42d1d52040cab22142f15d53875a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:37.192706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:37.395206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31c9a3-74cf-464c-b2b0-2ea23b2a68d7 tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:37.437685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1c42d1d52040cab22142f15d53875a', 'request_id': 'req-a61739e3-21b9-48ce-972c-1fd94016086d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:37.477488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:37.484457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1c42d1d52040cab22142f15d53875a', 'request_id': 'req-a61739e3-21b9-48ce-972c-1fd94016086d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:37.507850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1c42d1d52040cab22142f15d53875a', 'request_id': 'req-a61739e3-21b9-48ce-972c-1fd94016086d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:37.555367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] The token cache is being invalidate because project 8a1c42d1d52040cab22142f15d53875a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:37.555676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:37.556464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:37.563330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 931/1805] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:36 2022] DELETE /identity/v3/projects/8a1c42d1d52040cab22142f15d53875a => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:37.563330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:37.563330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:37.883991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:37.884729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:37.897361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:37.897746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:37.900350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:37.907897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61739e3-21b9-48ce-972c-1fd94016086d tempest-VolumeQuotaClassesTest-1581780447 tempest-VolumeQuotaClassesTest-1581780447-project-admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:37.928838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:37.936998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:37.937687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:37.941080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:37.943582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:37.948400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:38.012722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:38.023158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:38.058644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:38.079607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 875/1806] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 1232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:38.117071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 932/1807] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:37 2022] POST /identity/v3/users => generated 311 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:38.118065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:38.162891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecfff2-ffed-46db-9989-9125e742c8e7 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:38.219635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:38.220931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:38.221446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:38.221931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:38.223519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:38.263826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:38.386477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 876/1808] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:38.388419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:38.464749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:38.469298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:38.476080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914de26-2acb-4275-870f-97c4277fef2c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:38.538743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:38.538743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:38.538743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:38.541961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] PATH_INFO: `/v3/projects/e55ed2d8581b47c09b99414ba4a0746f/users/146de29573cd4bc288f8c4e0ccd6a394/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:38.544391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8ca8d-e630-4f98-a2f1-f5187ea9b4cc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:38.605041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:38.606289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:38.607949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:38.607949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] PATH_INFO: `/v3/users/60e9edd5bf324e218c72e81819dff5f3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:38.620484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e9edd5bf324e218c72e81819dff5f3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:38.629908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:38.643657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55ed2d8581b47c09b99414ba4a0746f, user_id=146de29573cd4bc288f8c4e0ccd6a394, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:38.648893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:38.749400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 933/1809] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:38 2022] PUT /identity/v3/projects/e55ed2d8581b47c09b99414ba4a0746f/users/146de29573cd4bc288f8c4e0ccd6a394/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:38.753062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:38.959410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ef26-cc31-4e11-86ce-9acffbee204b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:38.998788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:39.000725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:39.001557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:39.002304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:39.004012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:39.025973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:39.028586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e9edd5bf324e218c72e81819dff5f3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:39.029857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e9edd5bf324e218c72e81819dff5f3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:39.099062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e9edd5bf324e218c72e81819dff5f3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:39.134191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 934/1810] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:39.140830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e9edd5bf324e218c72e81819dff5f3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:39.143910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:39.146305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26ee3e90-e310-4881-8ac7-67977842aaac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e9edd5bf324e218c72e81819dff5f3'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:39.221143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 877/1811] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:38 2022] DELETE /identity/v3/users/60e9edd5bf324e218c72e81819dff5f3 => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:39.231168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a912e0-23db-4ba4-91a9-56e91d60ca5d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:39.231629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a912e0-23db-4ba4-91a9-56e91d60ca5d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:39.234441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a912e0-23db-4ba4-91a9-56e91d60ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:39.296810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d486b-6849-49e0-8fb8-217df6cbe6b5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:39.374939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f3a912e0-23db-4ba4-91a9-56e91d60ca5d None None] MFA Rules not processed for user `4294d6dff5054da7afcb37b8b28f1a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:39.384883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f3a912e0-23db-4ba4-91a9-56e91d60ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:39.664171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:39.893704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd184b22-1e01-4517-8313-5ca0e748feeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:39.894853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd184b22-1e01-4517-8313-5ca0e748feeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:39.895450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd184b22-1e01-4517-8313-5ca0e748feeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:39.906631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd184b22-1e01-4517-8313-5ca0e748feeb admin admin] PATH_INFO: `/v3/projects/e55ed2d8581b47c09b99414ba4a0746f/users/146de29573cd4bc288f8c4e0ccd6a394/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:39.937033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd184b22-1e01-4517-8313-5ca0e748feeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55ed2d8581b47c09b99414ba4a0746f, user_id=146de29573cd4bc288f8c4e0ccd6a394, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:39.939374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 878/1812] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 718 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:39.970533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6c563-043f-4850-bd3d-ca89590fd8f6 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:39.974095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd184b22-1e01-4517-8313-5ca0e748feeb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:39.976077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6c563-043f-4850-bd3d-ca89590fd8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:39.976543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6c563-043f-4850-bd3d-ca89590fd8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:40.101710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 935/1813] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:39 2022] PUT /identity/v3/projects/e55ed2d8581b47c09b99414ba4a0746f/users/146de29573cd4bc288f8c4e0ccd6a394/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:40.114674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:40.115178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:40.115660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:40.140037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c8a6c563-043f-4850-bd3d-ca89590fd8f6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:40.161863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c8a6c563-043f-4850-bd3d-ca89590fd8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:40.237633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] MFA Rules not processed for user `76c1fab92b854cd8937cc80818159191`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:40.241906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:40.599277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 879/1814] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 638 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:40.604655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:40.608853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:40.610532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:40.638306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 936/1815] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:40.648971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:40.654071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:40.707473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:40.814657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] MFA Rules not processed for user `146de29573cd4bc288f8c4e0ccd6a394`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:40.821774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:40.968197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfe4da-1829-404d-a021-ae915236ae30 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:40.996379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:41.006220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:41.007507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:41.009457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:41.044305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:41.047442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:41.114002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 937/1816] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:41.138837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 880/1817] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:40 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:41.158320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:41.160052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:41.427375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9edf55-5ad3-405a-8260-36dc0a21611b service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:41.443119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:41.443886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:41.444170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:41.444446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:41.476417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:41.485579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:41.518051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 938/1818] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:42.091076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:42.106178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e867437-c466-41f4-9e98-8c215ba3a960 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:42.120129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:42.134676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:42.135653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:42.137636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:42.156774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:42.417928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:42.422014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:42.446946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 881/1819] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:42.677183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:42.678965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:42.726589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acfab8-1084-45ea-bcd1-6716950f3a1b service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:42.735395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:42.737002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:42.737624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:42.738246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] PATH_INFO: `/v3/projects/0ab903b3452f4fefa703328a1dceb1b8` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:42.740744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab903b3452f4fefa703328a1dceb1b8)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:42.744909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:42.815210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:42.866998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab903b3452f4fefa703328a1dceb1b8', 'request_id': 'req-a17c8791-e743-43eb-933a-e1a424d4bc43'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:42.882943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab903b3452f4fefa703328a1dceb1b8', 'request_id': 'req-a17c8791-e743-43eb-933a-e1a424d4bc43'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:42.904188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab903b3452f4fefa703328a1dceb1b8', 'request_id': 'req-a17c8791-e743-43eb-933a-e1a424d4bc43'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:42.924044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] The token cache is being invalidate because project 0ab903b3452f4fefa703328a1dceb1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:42.924283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:42.924944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:42.927566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 939/1820] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:42 2022] DELETE /identity/v3/projects/0ab903b3452f4fefa703328a1dceb1b8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:39:42.932428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:43.071062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1de966-d9e5-4161-b2ff-e5f277bdc5a5 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:43.188167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:43.189179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:43.189368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:43.189697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] PATH_INFO: `/v3/users/d563005cae6d444c9068fc8428dcc092` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:43.191835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d563005cae6d444c9068fc8428dcc092)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:43.194968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:43.250673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:43.271787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd563005cae6d444c9068fc8428dcc092'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:43.272778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd563005cae6d444c9068fc8428dcc092'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:43.295391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd563005cae6d444c9068fc8428dcc092'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:43.305521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd563005cae6d444c9068fc8428dcc092'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:43.306961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ce80e56-c12d-4164-b951-a05b29d25478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd563005cae6d444c9068fc8428dcc092'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:43.350784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 882/1821] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:42 2022] DELETE /identity/v3/users/d563005cae6d444c9068fc8428dcc092 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:44.387582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:44.389383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:44.406818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c8791-e743-43eb-933a-e1a424d4bc43 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:44.500670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:44.501655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:44.502046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:44.502413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] PATH_INFO: `/v3/projects/2585b4585ef84b07aa160d792cc4d491` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:44.504068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2585b4585ef84b07aa160d792cc4d491)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:44.507740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:44.578690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2585b4585ef84b07aa160d792cc4d491', 'request_id': 'req-8207c10c-5896-497c-9f81-589c85ca603c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:44.594010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2585b4585ef84b07aa160d792cc4d491', 'request_id': 'req-8207c10c-5896-497c-9f81-589c85ca603c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:44.636720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2585b4585ef84b07aa160d792cc4d491', 'request_id': 'req-8207c10c-5896-497c-9f81-589c85ca603c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:44.667706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] The token cache is being invalidate because project 2585b4585ef84b07aa160d792cc4d491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:44.668120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:44.670189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8207c10c-5896-497c-9f81-589c85ca603c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:44.673186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 940/1822] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:44 2022] DELETE /identity/v3/projects/2585b4585ef84b07aa160d792cc4d491 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:44.678695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:44.679562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:44.679688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:44.776834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:44.779435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:44.937508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 883/1823] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:44.949488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:44.949846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:44.950196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:44.975038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:44.976401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:44.996655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 941/1824] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:45.000714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:45.017185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e35c4a-7d70-468c-bf6a-b509bb0d5728 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:45.022739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:45.024995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.025532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.025802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.026536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:45.026845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:45.028857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:45.044617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 884/1825] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:39:45.049459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:45.067467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567c93-7f98-4d70-94c3-0e8223426f52 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:45.076250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b5a67-02de-439a-b571-c074b55caf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:45.077772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b5a67-02de-439a-b571-c074b55caf3e admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.078260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b5a67-02de-439a-b571-c074b55caf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.078752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b5a67-02de-439a-b571-c074b55caf3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.080009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b5a67-02de-439a-b571-c074b55caf3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:45.084178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b5a67-02de-439a-b571-c074b55caf3e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:45.149742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 942/1826] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:45 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:45.153742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:45.181760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82799b-132a-4f69-b8f6-aa1afc5be2c7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:45.325944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.326238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.329214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.379396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:45.380962 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.381137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] MFA Rules not processed for user `1ac7d3acf1204a9f98f9ee14c2b46a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:45.382287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.382898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.384175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:45.384466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:45.389659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:45.411981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:45.465807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 885/1827] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:45 2022] POST /identity/v3/users => generated 291 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:39:45.470750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:45.490211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3351902-66b1-4fa1-b0da-38fe187548d6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:45.501273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:45.502364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.502861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.503435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.505894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:45.509421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:45.533410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 886/1828] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:45.537139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:45.545928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 943/1829] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:45.562559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a503af-5a2a-4ac2-bac9-e750df5c0ac6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:45.569626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ffac7-f6e8-4a67-af8d-bd1695ec3b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:45.570444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ffac7-f6e8-4a67-af8d-bd1695ec3b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.570554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ffac7-f6e8-4a67-af8d-bd1695ec3b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.570805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ffac7-f6e8-4a67-af8d-bd1695ec3b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.571519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ffac7-f6e8-4a67-af8d-bd1695ec3b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:45.573566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ffac7-f6e8-4a67-af8d-bd1695ec3b51 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:45.586334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 887/1830] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:45.590641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:45.603227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f52e7c-8dbf-42a8-a603-6cebcd249591 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:45.608533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:45.609564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.609847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.610119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] PATH_INFO: `/v3/projects/96d18e5ca3db4a32aa2f944ae82556ca/users/706884336e964526ad41750a2f06e9a0/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.632462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d18e5ca3db4a32aa2f944ae82556ca, user_id=706884336e964526ad41750a2f06e9a0, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:45.637062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:45.657649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 944/1831] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:45 2022] PUT /identity/v3/projects/96d18e5ca3db4a32aa2f944ae82556ca/users/706884336e964526ad41750a2f06e9a0/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:45.677511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:45.677511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:45.677511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:45.689347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:45.718697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] MFA Rules not processed for user `706884336e964526ad41750a2f06e9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:45.720714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:45.872725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 888/1832] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:45.878696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.011894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141f457-5420-4a8a-9520-c955ad1a7226 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:46.020159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:46.022314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.022896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.023452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.024832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:46.028759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:46.083430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 945/1833] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:45 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:46.089898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.112667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b1888-e450-41f4-aa2d-7a95a9bec902 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:46.272674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:46.273333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.273610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.273880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.274986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:46.277283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:46.333743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 889/1834] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:46 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:46.338806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.351434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bb131-5705-4841-9f79-0e9250f40a3b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:46.357630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:46.358388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.358828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.359153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.360004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:46.361781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:46.389958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 946/1835] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:46.393905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.405720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3a580-e1e0-40c4-aeeb-fa9dc9343238 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:46.419489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:46.420324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.420721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.420957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] PATH_INFO: `/v3/projects/04b7eee30b1242fe92708ae32ecdc22b/users/a9cdec43f5a04c13b951eb4524aef725/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.457507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b7eee30b1242fe92708ae32ecdc22b, user_id=a9cdec43f5a04c13b951eb4524aef725, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:46.464729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:46.507906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 890/1836] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:46 2022] PUT /identity/v3/projects/04b7eee30b1242fe92708ae32ecdc22b/users/a9cdec43f5a04c13b951eb4524aef725/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:46.509910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.529919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025697-91c7-4f14-99aa-2cf30cdb53e5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:46.617224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbaa75-3290-4abe-9b9b-d3f087241ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:46.618182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbaa75-3290-4abe-9b9b-d3f087241ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.618565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbaa75-3290-4abe-9b9b-d3f087241ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.618939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbaa75-3290-4abe-9b9b-d3f087241ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.621268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbaa75-3290-4abe-9b9b-d3f087241ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:46.621824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbaa75-3290-4abe-9b9b-d3f087241ab7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:46.633314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 947/1837] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:46.638395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.654143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c80976-4805-4cfc-a65a-8c46a2fc6983 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:46.660216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:46.661176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.661525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.661896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] PATH_INFO: `/v3/projects/04b7eee30b1242fe92708ae32ecdc22b/users/a9cdec43f5a04c13b951eb4524aef725/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.664500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b7eee30b1242fe92708ae32ecdc22b, user_id=a9cdec43f5a04c13b951eb4524aef725, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:46.667222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:46.695635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 891/1838] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:46 2022] PUT /identity/v3/projects/04b7eee30b1242fe92708ae32ecdc22b/users/a9cdec43f5a04c13b951eb4524aef725/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:46.703621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:46.703942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:46.704218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:46.719117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:46.740234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] MFA Rules not processed for user `a9cdec43f5a04c13b951eb4524aef725`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:46.741662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:46.875168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 948/1839] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:46.888447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:46.890634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:47.076655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:47.100098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce06d-9d5d-43e7-8156-6710c4ffaacd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:47.131704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:47.132881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:47.133348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:47.133831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:47.191862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:47.198501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:47.274936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 892/1840] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:47.610911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:47.614008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:47.634285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fffa3-ae07-458b-94ef-49439c3c1917 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:47.639911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fbbdf-2913-4d24-87c5-5bcf4366f93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:47.641350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbbdf-2913-4d24-87c5-5bcf4366f93a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:47.641750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbbdf-2913-4d24-87c5-5bcf4366f93a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:47.642077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbbdf-2913-4d24-87c5-5bcf4366f93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:47.742690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbbdf-2913-4d24-87c5-5bcf4366f93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:47.745579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbbdf-2913-4d24-87c5-5bcf4366f93a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:47.776082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 949/1841] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:52.339813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:52.511963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910eaac-c79a-491b-8f08-046a8a7ac170 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:52.519140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:52.522648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:52.522648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:52.522648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] PATH_INFO: `/v3/users/706884336e964526ad41750a2f06e9a0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:52.523691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=706884336e964526ad41750a2f06e9a0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:52.526354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:52.690609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706884336e964526ad41750a2f06e9a0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:52.691900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706884336e964526ad41750a2f06e9a0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:52.788493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706884336e964526ad41750a2f06e9a0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:52.802403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706884336e964526ad41750a2f06e9a0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:52.803953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706884336e964526ad41750a2f06e9a0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:52.835831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 893/1842] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:52 2022] DELETE /identity/v3/users/706884336e964526ad41750a2f06e9a0 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:52.840662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:52.842749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:52.852343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:52.900483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:52.902976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:53.170422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 950/1843] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:53.210271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.213312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:53.351524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f057cb-bf1e-4420-bdf3-1024eddb9cec admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.356358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3753d-7218-4557-bee0-bfcd2752dab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.357305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3753d-7218-4557-bee0-bfcd2752dab1 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.357434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3753d-7218-4557-bee0-bfcd2752dab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.357736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3753d-7218-4557-bee0-bfcd2752dab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.369631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3753d-7218-4557-bee0-bfcd2752dab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.372129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3753d-7218-4557-bee0-bfcd2752dab1 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.387680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 894/1844] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:53.538856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.543089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:53.570874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d02ef6-3f89-459d-8849-a0f246096fa9 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.581713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.583078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.583867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.584490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] PATH_INFO: `/v3/users/82ccfa7249b74b6b8a471d8f964c339b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.587227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82ccfa7249b74b6b8a471d8f964c339b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.591505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.701563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:53.742382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82ccfa7249b74b6b8a471d8f964c339b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:53.743485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82ccfa7249b74b6b8a471d8f964c339b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:53.762567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82ccfa7249b74b6b8a471d8f964c339b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:53.795308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82ccfa7249b74b6b8a471d8f964c339b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:53.810648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82ccfa7249b74b6b8a471d8f964c339b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:53.824604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 951/1845] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:53 2022] DELETE /identity/v3/users/82ccfa7249b74b6b8a471d8f964c339b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:53.830459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a414-8272-4f4e-87d7-db40957e7b75 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.835852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a414-8272-4f4e-87d7-db40957e7b75 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.836534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a414-8272-4f4e-87d7-db40957e7b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.906947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6849a414-8272-4f4e-87d7-db40957e7b75 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:53.908381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6849a414-8272-4f4e-87d7-db40957e7b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:53.916912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:54.085762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 895/1846] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:54.110036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:54.111801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:54.248530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b25f15-0984-48e4-9e5f-b279639431b1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:54.253447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd07a2f-56ca-46d1-8671-8889611b5cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:54.254819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd07a2f-56ca-46d1-8671-8889611b5cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.255538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd07a2f-56ca-46d1-8671-8889611b5cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.256182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd07a2f-56ca-46d1-8671-8889611b5cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.278344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd07a2f-56ca-46d1-8671-8889611b5cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:54.279842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd07a2f-56ca-46d1-8671-8889611b5cf7 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:54.309015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 952/1847] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:54.571429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37fd5e-559d-49d9-81a1-f8cb602e0807 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.572250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37fd5e-559d-49d9-81a1-f8cb602e0807 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.572250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37fd5e-559d-49d9-81a1-f8cb602e0807 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.575699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 896/1848] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:39:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:54.594703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.595144 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.595611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.714473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:54.718599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:54.897408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37fd5e-559d-49d9-81a1-f8cb602e0807 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:54.902544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cf37fd5e-559d-49d9-81a1-f8cb602e0807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:55.007031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37fd5e-559d-49d9-81a1-f8cb602e0807 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:55.027019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:55.029211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:55.032992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:55.033822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] PATH_INFO: `/v3/projects/96d18e5ca3db4a32aa2f944ae82556ca` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:55.054825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d18e5ca3db4a32aa2f944ae82556ca)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:55.060545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:55.204317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 953/1849] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 612 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:55.268162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d18e5ca3db4a32aa2f944ae82556ca', 'request_id': 'req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.282060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d18e5ca3db4a32aa2f944ae82556ca', 'request_id': 'req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.299524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d18e5ca3db4a32aa2f944ae82556ca', 'request_id': 'req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.328029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] The token cache is being invalidate because project 96d18e5ca3db4a32aa2f944ae82556ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:55.328553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.329809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.336492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 897/1850] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:54 2022] DELETE /identity/v3/projects/96d18e5ca3db4a32aa2f944ae82556ca => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:55.338761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:55.368089 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f36dc-e2f5-4514-9295-0d362bf8df4e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:55.454307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:55.507342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:55.508124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:55.508476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:55.509028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] PATH_INFO: `/v3/users/a9cdec43f5a04c13b951eb4524aef725` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:55.511302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9cdec43f5a04c13b951eb4524aef725)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:55.515349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:55.659164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9cdec43f5a04c13b951eb4524aef725'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.661071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9cdec43f5a04c13b951eb4524aef725'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.702448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9cdec43f5a04c13b951eb4524aef725'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.739927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9cdec43f5a04c13b951eb4524aef725'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.778333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9cdec43f5a04c13b951eb4524aef725'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:55.810519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 954/1851] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:55 2022] DELETE /identity/v3/users/a9cdec43f5a04c13b951eb4524aef725 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:39:55.834577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:55.836669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:56.049302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef33d7-6117-4046-9bc0-a2d1ad1d325b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:56.149664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:56.242823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:56.245340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:56.245832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:56.249011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:56.253586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:56.290164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:56.297674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:56.334514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 898/1852] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:39:57.399125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.401048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:57.501813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be8af-2a97-40d2-93d9-2c514a00a8f5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.508321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.509100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.509823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.510489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] PATH_INFO: `/v3/projects/04b7eee30b1242fe92708ae32ecdc22b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.512489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b7eee30b1242fe92708ae32ecdc22b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.514947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.601637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b7eee30b1242fe92708ae32ecdc22b', 'request_id': 'req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.608010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.610197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:57.621574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b7eee30b1242fe92708ae32ecdc22b', 'request_id': 'req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.624184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:57.631578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbee4f-f8e2-44d8-a79f-6374b543467b service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.642232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b7eee30b1242fe92708ae32ecdc22b', 'request_id': 'req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.676005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] The token cache is being invalidate because project 04b7eee30b1242fe92708ae32ecdc22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:57.680162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.681017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-dbb26ec6-f5d4-4d68-b2d1-32c375e55287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.687831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 955/1853] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:57 2022] DELETE /identity/v3/projects/04b7eee30b1242fe92708ae32ecdc22b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:57.689096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff413-288a-4795-81cd-66a3a4daa974 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.689457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff413-288a-4795-81cd-66a3a4daa974 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.689805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff413-288a-4795-81cd-66a3a4daa974 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.691804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 956/1854] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:57.695355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5110b8-50f3-4533-ab12-e03ccd0ea255 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.695355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5110b8-50f3-4533-ab12-e03ccd0ea255 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.695594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5110b8-50f3-4533-ab12-e03ccd0ea255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.742551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.743555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.743832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.744057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ed5110b8-50f3-4533-ab12-e03ccd0ea255 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:57.744360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] PATH_INFO: `/v3/projects/55db4c94326e4b37bd2643eb792df566` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.745840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55db4c94326e4b37bd2643eb792df566)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.746101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ed5110b8-50f3-4533-ab12-e03ccd0ea255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:57.748231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.757237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 957/1855] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:57.765022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb103fb-ff66-441e-8522-bf2fd0412ef7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.766420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb103fb-ff66-441e-8522-bf2fd0412ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.767183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb103fb-ff66-441e-8522-bf2fd0412ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.800641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ccb103fb-ff66-441e-8522-bf2fd0412ef7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:57.803743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ccb103fb-ff66-441e-8522-bf2fd0412ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:57.845872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55db4c94326e4b37bd2643eb792df566', 'request_id': 'req-abd91048-43fa-4af2-a19a-9efaf24bb17d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.873539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55db4c94326e4b37bd2643eb792df566', 'request_id': 'req-abd91048-43fa-4af2-a19a-9efaf24bb17d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.898057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55db4c94326e4b37bd2643eb792df566', 'request_id': 'req-abd91048-43fa-4af2-a19a-9efaf24bb17d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.899409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 958/1856] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:57.901568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.902915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.903261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.922882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] The token cache is being invalidate because project 55db4c94326e4b37bd2643eb792df566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:57.923419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.924809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:57.927900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 899/1857] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:57 2022] DELETE /identity/v3/projects/55db4c94326e4b37bd2643eb792df566 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:57.929507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.932034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:57.937417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:57.940573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:57.958360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd91048-43fa-4af2-a19a-9efaf24bb17d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.968243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.969442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.970006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.970560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.972005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:57.972609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.976461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.003239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 900/1858] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:58.017148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.020733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:58.058050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c5fc4-a409-425d-99d7-8a080a1cb3cd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.069694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.071268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.072383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.072838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] PATH_INFO: `/v3/users/146de29573cd4bc288f8c4e0ccd6a394` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.075513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146de29573cd4bc288f8c4e0ccd6a394)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.080084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.097458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 959/1859] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:39:58.100352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.120351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a56d0c-1f75-41b6-a960-b187d0961414 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.136993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.137362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.137645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.138030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.138593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.140688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.185126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:58.236938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146de29573cd4bc288f8c4e0ccd6a394'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:58.238812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146de29573cd4bc288f8c4e0ccd6a394'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:58.258324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 960/1860] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:58 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:58.264366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.288778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146de29573cd4bc288f8c4e0ccd6a394'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:58.323797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146de29573cd4bc288f8c4e0ccd6a394'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:58.328341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146de29573cd4bc288f8c4e0ccd6a394'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:58.384760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 901/1861] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:39:58 2022] DELETE /identity/v3/users/146de29573cd4bc288f8c4e0ccd6a394 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:58.401218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.403673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:58.409959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:58.434585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446128bd-5cb0-47de-8ec6-bb3cec085534 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.680715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f131-616b-49ac-9582-c0120624b861 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.695658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.696069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.696643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.696920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.697182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.697333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.697732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.698161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.699840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.702049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:58.705664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.722396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.733596 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.772313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 961/1862] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:58 2022] POST /identity/v3/users => generated 284 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:58.773340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.776670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 902/1863] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:39:58.778382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:58.812432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599071ca-59bd-414c-b934-6835d745e95c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.821738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.822779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.823243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.823790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.826227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.829855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.852667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 962/1864] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:39:58.858342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.888326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7b1b-bc62-48dc-9eb4-934a303177c2 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.898000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.899245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.899575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.899888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.900936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.903435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.922532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 903/1865] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:58.927095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.940583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.946258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:58.957233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3750c7-b605-44c7-937f-8f2ab183b198 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.969986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506718de-85d4-49f7-a7b4-65efd31b9757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.971832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506718de-85d4-49f7-a7b4-65efd31b9757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.972449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506718de-85d4-49f7-a7b4-65efd31b9757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.973206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506718de-85d4-49f7-a7b4-65efd31b9757 admin admin] PATH_INFO: `/v3/projects/30ca5c91fd1542ffabc9e739b1d3e051/users/db64e8eb35aa425397067352cf20a7e7/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.980560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822f71b-a6e8-4c41-9b74-703816f183f4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.989827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.990888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.991367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.991908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] PATH_INFO: `/v3/projects/e55ed2d8581b47c09b99414ba4a0746f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.994287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55ed2d8581b47c09b99414ba4a0746f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.998723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.032964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506718de-85d4-49f7-a7b4-65efd31b9757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ca5c91fd1542ffabc9e739b1d3e051, user_id=db64e8eb35aa425397067352cf20a7e7, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.046271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506718de-85d4-49f7-a7b4-65efd31b9757 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.063464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 963/1866] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:58 2022] PUT /identity/v3/projects/30ca5c91fd1542ffabc9e739b1d3e051/users/db64e8eb35aa425397067352cf20a7e7/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:59.075505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.075828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.076104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.085700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55ed2d8581b47c09b99414ba4a0746f', 'request_id': 'req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:59.100264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55ed2d8581b47c09b99414ba4a0746f', 'request_id': 'req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:59.113742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55ed2d8581b47c09b99414ba4a0746f', 'request_id': 'req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:59.114042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] MFA Rules not processed for user `db64e8eb35aa425397067352cf20a7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:59.115621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:59.130479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] The token cache is being invalidate because project e55ed2d8581b47c09b99414ba4a0746f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:39:59.130950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:59.131749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f1fe6ab7-22d7-406d-a9a4-fc0e0f6a6139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:39:59.133600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 904/1867] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:39:58 2022] DELETE /identity/v3/projects/e55ed2d8581b47c09b99414ba4a0746f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:59.141591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c0fa7-0a93-4f67-be50-d544c73f0060 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.141978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c0fa7-0a93-4f67-be50-d544c73f0060 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.142352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c0fa7-0a93-4f67-be50-d544c73f0060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.152336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:59.165429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-612c0fa7-0a93-4f67-be50-d544c73f0060 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:59.167684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-612c0fa7-0a93-4f67-be50-d544c73f0060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:59.245126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 964/1868] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:59.249366 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.251851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:59.266180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 905/1869] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:59.275405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.275686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.276002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.277843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b2579-c493-4bcb-8b4e-4d5ae33f386a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.282671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.283392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.283885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.284216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.286739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.288853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.304149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:59.305920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:59.324961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 906/1870] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:39:59.333390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.335190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 965/1871] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:59.336267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.344460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acd421-29d2-4099-915a-a11bdbc5517e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.366248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:59.432116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d204647-75b8-45f9-aa81-30fde4efadfa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.438095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.438902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.439199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.439485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.440264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.442535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.442850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.443183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.443451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.443722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.444603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:59.444845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.447052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.462823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 907/1872] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:59.467300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.473997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 966/1873] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:59.478597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.483964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c1edf-f2f7-4574-8e27-9b5f35c151a8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.491385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b928e2-7d2a-46a0-8867-494a41a1118c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.492539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.493264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.493616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.493962 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.494749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.496735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.497557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.497908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.498092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.498092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.498803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.500515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.524531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 967/1874] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:59.530142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.550275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 908/1875] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:59.550493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad66f-80ef-4ed7-8c42-8d9f68989b7e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.553852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.658669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.659513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.660007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.660669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] PATH_INFO: `/v3/projects/83eb374fbdcc40bbae715ee3112d063e/users/1f1b767716d74a578dcbc535ab7f7325/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.663195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dca84-8e47-4928-bf29-181abad30300 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.669957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.670747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.671128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.671517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.672338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.674415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.689647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83eb374fbdcc40bbae715ee3112d063e, user_id=1f1b767716d74a578dcbc535ab7f7325, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.695526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.701924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:59.712911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 909/1876] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:59.717276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 968/1877] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:59 2022] PUT /identity/v3/projects/83eb374fbdcc40bbae715ee3112d063e/users/1f1b767716d74a578dcbc535ab7f7325/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:59.718431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.722220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.732365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f608d-65e3-4a94-b4dc-177eee11bfbd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.857917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01692bf-acd7-48d8-acea-abc003caf303 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.865950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.866752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.867219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.867590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.868514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.870290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.876464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.877575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.878231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.879008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.880538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.883715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.904548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 969/1878] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:59.906378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.907791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 910/1879] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:39:59.919292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:59.922168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae92cfd-3207-422d-a4fd-1873618a6de2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.929728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcedd8-d33e-4a04-a733-f1838b1b31b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:59.930450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcedd8-d33e-4a04-a733-f1838b1b31b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.930798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcedd8-d33e-4a04-a733-f1838b1b31b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.931080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcedd8-d33e-4a04-a733-f1838b1b31b2 admin admin] PATH_INFO: `/v3/projects/83eb374fbdcc40bbae715ee3112d063e/users/1f1b767716d74a578dcbc535ab7f7325/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.933499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcedd8-d33e-4a04-a733-f1838b1b31b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83eb374fbdcc40bbae715ee3112d063e, user_id=1f1b767716d74a578dcbc535ab7f7325, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:59.936622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcedd8-d33e-4a04-a733-f1838b1b31b2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:59.942850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c718d5c-dc1c-4e13-abf0-339d9258ba33 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:59.962525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 970/1880] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:39:59 2022] PUT /identity/v3/projects/83eb374fbdcc40bbae715ee3112d063e/users/1f1b767716d74a578dcbc535ab7f7325/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:39:59.964970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:59.965510 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:59.966037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:59.978154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:00.007750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] MFA Rules not processed for user `1f1b767716d74a578dcbc535ab7f7325`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:00.009435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:00.058125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.058790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.059061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.060108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.060849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:00.062979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:00.077900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 911/1881] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:00.082843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.099838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38db1-5ef6-4aeb-9987-f802ab1daf07 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:00.105923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8a30d-6b00-4807-b113-fe5859c862c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.106724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a30d-6b00-4807-b113-fe5859c862c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.107075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a30d-6b00-4807-b113-fe5859c862c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.107417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a30d-6b00-4807-b113-fe5859c862c2 admin admin] PATH_INFO: `/v3/projects/f6819a2a7cd54418a324631c5a7d72e6/users/01b384787db64df68691db7fd0b59416/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.129316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8a30d-6b00-4807-b113-fe5859c862c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6819a2a7cd54418a324631c5a7d72e6, user_id=01b384787db64df68691db7fd0b59416, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:00.132251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8a30d-6b00-4807-b113-fe5859c862c2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:00.155455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 912/1882] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:00 2022] PUT /identity/v3/projects/f6819a2a7cd54418a324631c5a7d72e6/users/01b384787db64df68691db7fd0b59416/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:00.168671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.168671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.168671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.208559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] MFA Rules not processed for user `01b384787db64df68691db7fd0b59416`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:00.209916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:00.220377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 971/1883] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:00.222946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.230369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:00.334622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:00.347373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 913/1884] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:00.353332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.405497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b38d59-29d4-4d89-9221-113f959f7457 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:00.522248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab8569-dca9-4cf2-8848-e489a731ba20 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:00.531391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.532349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.532875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.533321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.534054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:00.536667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:00.553809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.555239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.556446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.557208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.613675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 914/1885] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:00 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:00.614922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.722247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:00.728259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:00.731472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:00.760788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 972/1886] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:00.811369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e4d78-0325-4756-9dc8-8b17339a4fb3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:00.820019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.822168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.822168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.823127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.824071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:00.826578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:00.871673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 915/1887] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:00 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:00.875981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.892570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c312ad-09d1-4b2a-a213-24aed89b1352 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:00.917420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.918286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.918670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.919047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.920854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:00.923581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:00.934898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.967209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 973/1888] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:00.971869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.004883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7390431-09ef-4d3b-8413-c218320b8025 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.020652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.023715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.024311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.026709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] PATH_INFO: `/v3/projects/d3cf241bca0e440ba5264b0ed37608bd/users/7c8f8c324f214eae9129238a8b672794/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.061829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d9223-c8b8-4a62-992b-8c7c8d786be8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.066532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.067527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.067980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.068416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.087431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cf241bca0e440ba5264b0ed37608bd, user_id=7c8f8c324f214eae9129238a8b672794, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.090319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.112274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:01.125252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 974/1889] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:00 2022] PUT /identity/v3/projects/d3cf241bca0e440ba5264b0ed37608bd/users/7c8f8c324f214eae9129238a8b672794/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:01.126975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.165124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.169245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.195622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 916/1890] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:01.286291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e32636-888b-4621-89e8-bd3265b5dd29 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.297362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4917e-2417-4b6c-9c56-0c513710645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.298475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4917e-2417-4b6c-9c56-0c513710645a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.299037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4917e-2417-4b6c-9c56-0c513710645a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.299540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4917e-2417-4b6c-9c56-0c513710645a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.301271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4917e-2417-4b6c-9c56-0c513710645a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.306551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4917e-2417-4b6c-9c56-0c513710645a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.331604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 975/1891] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:01.333111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.346380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb00e4-beea-4e7f-8e72-eb02da2455a1 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.384403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.385683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.386151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.386622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] PATH_INFO: `/v3/projects/d3cf241bca0e440ba5264b0ed37608bd/users/7c8f8c324f214eae9129238a8b672794/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.390969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cf241bca0e440ba5264b0ed37608bd, user_id=7c8f8c324f214eae9129238a8b672794, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.403597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.434927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 917/1892] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:01 2022] PUT /identity/v3/projects/d3cf241bca0e440ba5264b0ed37608bd/users/7c8f8c324f214eae9129238a8b672794/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:01.451000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba24e-0231-418e-be19-677665c88baa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.451865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba24e-0231-418e-be19-677665c88baa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.452452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba24e-0231-418e-be19-677665c88baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.494985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-703ba24e-0231-418e-be19-677665c88baa None None] MFA Rules not processed for user `7c8f8c324f214eae9129238a8b672794`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:01.496523 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-703ba24e-0231-418e-be19-677665c88baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:01.659859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 976/1893] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:01.689345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.775842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3003b-fb3e-4397-8604-a8a12e4cceb3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.779540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36bf79-9b78-42dc-ae22-480f08b63308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.780239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36bf79-9b78-42dc-ae22-480f08b63308 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.780520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36bf79-9b78-42dc-ae22-480f08b63308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.780813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36bf79-9b78-42dc-ae22-480f08b63308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.855961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36bf79-9b78-42dc-ae22-480f08b63308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.858859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36bf79-9b78-42dc-ae22-480f08b63308 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.873710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 918/1894] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:03.799344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba24e-0231-418e-be19-677665c88baa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:03.801596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-703ba24e-0231-418e-be19-677665c88baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:03.872176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:03.906722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba24e-0231-418e-be19-677665c88baa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:03.914588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:03.915237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:03.915652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:03.915898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] PATH_INFO: `/v3/users/1b10ca42f1bb4be6a74603fa214831e5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:03.917446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b10ca42f1bb4be6a74603fa214831e5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:03.919494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:04.061224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b10ca42f1bb4be6a74603fa214831e5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:04.074639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b10ca42f1bb4be6a74603fa214831e5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:04.124409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b10ca42f1bb4be6a74603fa214831e5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:04.163051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b10ca42f1bb4be6a74603fa214831e5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:04.186410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b10ca42f1bb4be6a74603fa214831e5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:04.217956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 977/1895] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:03 2022] DELETE /identity/v3/users/1b10ca42f1bb4be6a74603fa214831e5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:04.226122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:04.228224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:04.228921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:04.295590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:04.297777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:04.464228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 919/1896] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:04.476778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:04.479994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:04.607573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2531e-85fd-45e8-995b-5ebe19f0f010 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:04.615472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1591439-fee2-4f9e-9e92-6e94aa154bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:04.616846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591439-fee2-4f9e-9e92-6e94aa154bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:04.617323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591439-fee2-4f9e-9e92-6e94aa154bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:04.617932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1591439-fee2-4f9e-9e92-6e94aa154bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:04.635717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1591439-fee2-4f9e-9e92-6e94aa154bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:04.638244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1591439-fee2-4f9e-9e92-6e94aa154bbb service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:04.651336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 978/1897] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:05.138919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.145199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:05.190013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c28b4-7a4a-43c1-97f8-1981b26c06f2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.211005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.216123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.218167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.219285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] PATH_INFO: `/v3/users/01b384787db64df68691db7fd0b59416` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.223874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b384787db64df68691db7fd0b59416)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.240383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.257195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:05.402314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b384787db64df68691db7fd0b59416'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:05.406148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b384787db64df68691db7fd0b59416'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:05.451155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b384787db64df68691db7fd0b59416'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:05.490575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b384787db64df68691db7fd0b59416'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:05.492798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b384787db64df68691db7fd0b59416'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:05.552222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 920/1898] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:05 2022] DELETE /identity/v3/users/01b384787db64df68691db7fd0b59416 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:05.561953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.562850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.562850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.586506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:05.587796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:05.725258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 979/1899] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:05.741432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.742969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:05.836228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33465183-623f-4cce-af1b-4aa2cf4ded44 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.850605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.854684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.855126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.868035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.901773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.906071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.934331 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 921/1900] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:06.053450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.055589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:06.068433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:06.075382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7a8b9-5c97-41e7-86c6-135ebe607d05 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.086197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.087464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.089384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.089384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] PATH_INFO: `/v3/projects/053023b614f34fa4a7fc6153fbdc986a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.090248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] RBAC: Authorizing `identity:delete_project(project_id=053023b614f34fa4a7fc6153fbdc986a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.095420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.213322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053023b614f34fa4a7fc6153fbdc986a', 'request_id': 'req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.232593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053023b614f34fa4a7fc6153fbdc986a', 'request_id': 'req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.258199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053023b614f34fa4a7fc6153fbdc986a', 'request_id': 'req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.328967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] The token cache is being invalidate because project 053023b614f34fa4a7fc6153fbdc986a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:06.328967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.330260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.346314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 980/1901] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:06 2022] DELETE /identity/v3/projects/053023b614f34fa4a7fc6153fbdc986a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:06.347618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.359561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01684b32-8800-47b7-9fd3-6d44de9debcb service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.564838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.567458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.568019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.570171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] PATH_INFO: `/v3/users/b0504a180e274d68961e3a1e81e21c79` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.573729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0504a180e274d68961e3a1e81e21c79)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.581065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.664833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:06.681456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.685619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:06.711429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46cd5-eb9c-4d50-b3ec-fcdc17e6adae admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.730769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0504a180e274d68961e3a1e81e21c79'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.732027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0504a180e274d68961e3a1e81e21c79'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.741578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.742285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.742573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.743003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] PATH_INFO: `/v3/projects/f6819a2a7cd54418a324631c5a7d72e6` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.746291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6819a2a7cd54418a324631c5a7d72e6)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.752141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.761979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0504a180e274d68961e3a1e81e21c79'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.780437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0504a180e274d68961e3a1e81e21c79'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.782799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0504a180e274d68961e3a1e81e21c79'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.824067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 922/1902] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:06 2022] DELETE /identity/v3/users/b0504a180e274d68961e3a1e81e21c79 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:06.841758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6819a2a7cd54418a324631c5a7d72e6', 'request_id': 'req-cbcb339d-9618-47be-b15b-eec13a9832ae'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.855006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6819a2a7cd54418a324631c5a7d72e6', 'request_id': 'req-cbcb339d-9618-47be-b15b-eec13a9832ae'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.874980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6819a2a7cd54418a324631c5a7d72e6', 'request_id': 'req-cbcb339d-9618-47be-b15b-eec13a9832ae'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.908946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] The token cache is being invalidate because project f6819a2a7cd54418a324631c5a7d72e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:06.909480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.912554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:06.917530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 981/1903] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:06 2022] DELETE /identity/v3/projects/f6819a2a7cd54418a324631c5a7d72e6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:06.924190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.928895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:07.217229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d19810-c878-4197-8777-202e7fd02880 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:07.232061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:07.233682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:07.234281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:07.234889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] PATH_INFO: `/v3/users/7c8f8c324f214eae9129238a8b672794` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:07.237753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c8f8c324f214eae9129238a8b672794)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:07.244169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:07.341503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c8f8c324f214eae9129238a8b672794'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.342564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c8f8c324f214eae9129238a8b672794'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.367736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c8f8c324f214eae9129238a8b672794'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.380696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c8f8c324f214eae9129238a8b672794'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.381634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c8f8c324f214eae9129238a8b672794'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.429809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 923/1904] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:06 2022] DELETE /identity/v3/users/7c8f8c324f214eae9129238a8b672794 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:07.499511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:07.501604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:07.733327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb339d-9618-47be-b15b-eec13a9832ae admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:07.755424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:07.822449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:07.823132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:07.823420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:07.823915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] PATH_INFO: `/v3/projects/ad42d2c255ba4cfc978603747f541904` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:07.825204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad42d2c255ba4cfc978603747f541904)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:07.827168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:07.890354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad42d2c255ba4cfc978603747f541904', 'request_id': 'req-16cc3609-b073-4d9f-88bf-223e311c94c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.893415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:07.895882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:07.903912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad42d2c255ba4cfc978603747f541904', 'request_id': 'req-16cc3609-b073-4d9f-88bf-223e311c94c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.927840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a3479-02da-45ee-b3f9-fd69a3407e6b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:07.928124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad42d2c255ba4cfc978603747f541904', 'request_id': 'req-16cc3609-b073-4d9f-88bf-223e311c94c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.949873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] The token cache is being invalidate because project ad42d2c255ba4cfc978603747f541904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:07.950765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.951183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-16cc3609-b073-4d9f-88bf-223e311c94c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:07.953072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 982/1905] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:07 2022] DELETE /identity/v3/projects/ad42d2c255ba4cfc978603747f541904 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:07.961703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe04d7-7943-4f8a-ba5e-7aa25f4bb9d7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:07.961703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe04d7-7943-4f8a-ba5e-7aa25f4bb9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:07.962263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe04d7-7943-4f8a-ba5e-7aa25f4bb9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:07.964865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:07.999729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fbfe04d7-7943-4f8a-ba5e-7aa25f4bb9d7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:08.001363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fbfe04d7-7943-4f8a-ba5e-7aa25f4bb9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:08.043059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.045738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.046192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.046641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] PATH_INFO: `/v3/projects/d3cf241bca0e440ba5264b0ed37608bd` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.049443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cf241bca0e440ba5264b0ed37608bd)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.052945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.144490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cf241bca0e440ba5264b0ed37608bd', 'request_id': 'req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:08.152567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cf241bca0e440ba5264b0ed37608bd', 'request_id': 'req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:08.164410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 983/1906] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:08.165973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cf241bca0e440ba5264b0ed37608bd', 'request_id': 'req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:08.166359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.166643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.166934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.180146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] The token cache is being invalidate because project d3cf241bca0e440ba5264b0ed37608bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:08.180451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:08.181582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b1dde8a6-b0b6-4727-a9a8-3587e57ae381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:08.183245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 924/1907] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:07 2022] DELETE /identity/v3/projects/d3cf241bca0e440ba5264b0ed37608bd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:08.195468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3a736-15ae-4030-9a8d-3f8d02425845 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.195982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3a736-15ae-4030-9a8d-3f8d02425845 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.196163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:08.196455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3a736-15ae-4030-9a8d-3f8d02425845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.197873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:08.231373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0fb3a736-15ae-4030-9a8d-3f8d02425845 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:08.237140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0fb3a736-15ae-4030-9a8d-3f8d02425845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:08.283552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:08.313748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 984/1908] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:08.320167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.337184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b51c10-c219-4593-8947-32be988f99ce None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.349508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.353364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.360507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.361417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.363373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:08.365158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.368432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.397539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 925/1909] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:08.400195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.401626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.402091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.408859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 985/1910] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:08.414252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.425863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff428c-fcee-4288-b0cc-6a91890aedb8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.434866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.435796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.436140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.436581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.437309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.439927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.446204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:08.448370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:08.468433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 926/1911] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:08.470115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.493996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883c9d4-f9e6-4639-a849-4e6f3c38ff03 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.504000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.505073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.505822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.506286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.507885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:08.508400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.512544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.516773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 986/1912] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:08.523270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.545086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 927/1913] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:08.546882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.568112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7628cea-607a-47ff-a1e2-038a2f151d0e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.633515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134fc6-6d82-4fc0-aa4d-08a79745b2ec admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.642022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.642253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.642686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.643038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.644320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.647067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.660467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:08.684040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 987/1914] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/users => generated 279 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:08.689848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.708542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632f1d6-cacb-4145-b132-9af4879d757d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.715055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.715961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.716499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.716829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.717885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.719706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.741746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 988/1915] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:08.743613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.756425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.758522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.758811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.759102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.759873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.763025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.769563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaaa09-5c0f-4eee-bb8e-1e3aff9a16a1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.778270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.779753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.780505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.781209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.782504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.785780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.803975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 928/1916] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:08.805305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.807591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:08.811999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 989/1917] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:08.813142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:08.834261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06d00-2a53-4084-a0b8-676c3be6b08a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.943157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502e8a-10c4-442d-9a28-09ba03588996 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:08.953148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.954146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:08.954313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:08.959407 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] PATH_INFO: `/v3/users/db64e8eb35aa425397067352cf20a7e7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:08.965754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db64e8eb35aa425397067352cf20a7e7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:08.967982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:08.998197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:08.999460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.000341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.002201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.006796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.016070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.099161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db64e8eb35aa425397067352cf20a7e7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:09.099795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db64e8eb35aa425397067352cf20a7e7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:09.126438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db64e8eb35aa425397067352cf20a7e7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:09.127329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 990/1918] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:08 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:09.129458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.137513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db64e8eb35aa425397067352cf20a7e7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:09.138975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db64e8eb35aa425397067352cf20a7e7'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:09.156211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bbe80-48ac-4f73-b1e9-4adfceef6d63 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.169097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 929/1919] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:08 2022] DELETE /identity/v3/users/db64e8eb35aa425397067352cf20a7e7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:09.172400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.176810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:09.180881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6342-804a-4c45-8cf2-012c0edbee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.183324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6342-804a-4c45-8cf2-012c0edbee80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.183614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6342-804a-4c45-8cf2-012c0edbee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.184368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6342-804a-4c45-8cf2-012c0edbee80 admin admin] PATH_INFO: `/v3/projects/af0c16f98948427caaf8d4f1c2e909e7/users/d5b413c7ba0d4836b3661b91b6431370/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.208605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.229290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac6342-804a-4c45-8cf2-012c0edbee80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0c16f98948427caaf8d4f1c2e909e7, user_id=d5b413c7ba0d4836b3661b91b6431370, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.238632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac6342-804a-4c45-8cf2-012c0edbee80 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.267492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 991/1920] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:09 2022] PUT /identity/v3/projects/af0c16f98948427caaf8d4f1c2e909e7/users/d5b413c7ba0d4836b3661b91b6431370/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:09.271544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.272285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.273392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.292702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.310260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:09.313542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:09.340990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547227-dc96-4e24-9973-7c34951a2c70 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.467471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d96e2a-de6b-40e3-8c2b-819685492239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.468157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96e2a-de6b-40e3-8c2b-819685492239 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.468412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96e2a-de6b-40e3-8c2b-819685492239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.468687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d96e2a-de6b-40e3-8c2b-819685492239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.469412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d96e2a-de6b-40e3-8c2b-819685492239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.471105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d96e2a-de6b-40e3-8c2b-819685492239 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.495215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 930/1921] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:09.496881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.497174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.497455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.522424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 992/1922] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:09.526494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.577530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5a981-18cb-493c-bc3f-af2bc2636400 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.592034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.593046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.593488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.593911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.595076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.598409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.603652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] MFA Rules not processed for user `d5b413c7ba0d4836b3661b91b6431370`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:09.604964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:09.615884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 993/1923] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:09.621783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.623742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:09.756873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 931/1924] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:09.758764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.766028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cccf-0032-4643-9045-85f85371e26a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.776941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.777732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.778072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.778410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.779178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9421-3f2a-457a-955c-0ad8c4c7d794 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.788486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4169a5-dc1f-4999-b815-77774843a3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.791215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4169a5-dc1f-4999-b815-77774843a3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.791877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.792552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4169a5-dc1f-4999-b815-77774843a3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.793702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4169a5-dc1f-4999-b815-77774843a3ce admin admin] PATH_INFO: `/v3/projects/019734069da44b2d88561549bd07544d/users/23a95d9f48c64b0fbb8879d361bc177d/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.793889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.809699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 994/1925] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:09.819306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.827590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:09.847417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4169a5-dc1f-4999-b815-77774843a3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=019734069da44b2d88561549bd07544d, user_id=23a95d9f48c64b0fbb8879d361bc177d, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.854608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4169a5-dc1f-4999-b815-77774843a3ce admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.859735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe91625-e7a1-43f6-b7ca-cfcf633f7fa0 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.871998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.872686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.872939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.875353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.878106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.883142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.904706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 932/1926] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:09 2022] PUT /identity/v3/projects/019734069da44b2d88561549bd07544d/users/23a95d9f48c64b0fbb8879d361bc177d/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:09.912638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.916570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.916849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.965812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 995/1927] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:09 2022] POST /identity/v3/projects => generated 357 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:09.968350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.006741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:10.009219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:10.155983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.157907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.166862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbba9c-f36a-41e5-a186-52e3ebb0f141 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.184377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.185856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.185856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.185856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.186416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.188339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.240899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 996/1928] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:09 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:10.249751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.256404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 933/1929] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:10.266854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e3021-cf43-4adc-a47d-35c4d6266721 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.272840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.273208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.274147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.274431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.274705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.275439 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.277153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.290709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 997/1930] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:10.296967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.321032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491159cb-6a2d-4846-b728-825ae77a0788 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.328011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.328894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.329312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.329732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] PATH_INFO: `/v3/projects/980c4a34c2ca4484bf806fd9abcd72fd/users/d1036c61174948a58bfbf3b1d8fa7b0d/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.375548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=980c4a34c2ca4484bf806fd9abcd72fd, user_id=d1036c61174948a58bfbf3b1d8fa7b0d, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.380859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.402067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 998/1931] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:10 2022] PUT /identity/v3/projects/980c4a34c2ca4484bf806fd9abcd72fd/users/d1036c61174948a58bfbf3b1d8fa7b0d/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:10.403136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.407588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:10.444302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5b255-0a0e-436e-b30d-63f44e2085e7 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.482425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b078-3b59-4078-b60d-0de1de1b4f1a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.487839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.488398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.488880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.489309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] PATH_INFO: `/v3/projects/30ca5c91fd1542ffabc9e739b1d3e051` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.492388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ca5c91fd1542ffabc9e739b1d3e051)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.493158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.547460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.549739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.550608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.551391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.576330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ca5c91fd1542ffabc9e739b1d3e051', 'request_id': 'req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:10.577121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.583399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.591996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ca5c91fd1542ffabc9e739b1d3e051', 'request_id': 'req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:10.605315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ca5c91fd1542ffabc9e739b1d3e051', 'request_id': 'req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:10.606638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 934/1932] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:10.608210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.627793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480aaf-e8b9-477e-b0ff-8b16384a3811 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.681307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] The token cache is being invalidate because project 30ca5c91fd1542ffabc9e739b1d3e051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:10.683013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:10.689885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:10.693865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 999/1933] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:10 2022] DELETE /identity/v3/projects/30ca5c91fd1542ffabc9e739b1d3e051 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:10.697358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.700187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:10.803785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.804455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.809003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.809392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.810341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.815242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.824621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.841541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 935/1934] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:10.850493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.855609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:10.877546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7b75-1d29-47f7-ad14-2af06ab6b01a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.886412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d32d24-4cdb-4c3b-b64c-6e2a92068a91 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.889862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.890761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.891479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.892384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] PATH_INFO: `/v3/users/1f1b767716d74a578dcbc535ab7f7325` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.894409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1b767716d74a578dcbc535ab7f7325)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.900175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.902061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae8fa2-af88-41f3-9ed6-9964c1e198b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.902787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae8fa2-af88-41f3-9ed6-9964c1e198b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.903153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae8fa2-af88-41f3-9ed6-9964c1e198b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.903437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae8fa2-af88-41f3-9ed6-9964c1e198b2 admin admin] PATH_INFO: `/v3/projects/980c4a34c2ca4484bf806fd9abcd72fd/users/d1036c61174948a58bfbf3b1d8fa7b0d/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.905965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae8fa2-af88-41f3-9ed6-9964c1e198b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980c4a34c2ca4484bf806fd9abcd72fd, user_id=d1036c61174948a58bfbf3b1d8fa7b0d, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.912652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae8fa2-af88-41f3-9ed6-9964c1e198b2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.944838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 936/1935] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:10 2022] PUT /identity/v3/projects/980c4a34c2ca4484bf806fd9abcd72fd/users/d1036c61174948a58bfbf3b1d8fa7b0d/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:10.981208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.981544 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.982485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:11.039377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1b767716d74a578dcbc535ab7f7325'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:11.045651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1b767716d74a578dcbc535ab7f7325'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:11.067339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1b767716d74a578dcbc535ab7f7325'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:11.078370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] MFA Rules not processed for user `d1036c61174948a58bfbf3b1d8fa7b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:11.080129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:11.099393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1b767716d74a578dcbc535ab7f7325'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:11.118116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1b767716d74a578dcbc535ab7f7325'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:11.129539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:11.135263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1000/1936] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:10 2022] DELETE /identity/v3/users/1f1b767716d74a578dcbc535ab7f7325 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:11.230709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 937/1937] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:11.247348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:11.249257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:11.348829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3868-b8a3-4bc6-927b-ff0df9afe0ef admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:11.359059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:11.363421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:11.364344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:11.365040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:11.533171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:11.538227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:11.564692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1001/1938] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:11.722632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:12.010424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78f7d7-af2f-4ce4-821c-e4c37ec0ef9c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:12.020221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:12.022618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:12.025953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:12.025953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:12.028656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:12.030293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:12.042796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:12.118496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:12.258018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:12.260490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b8252-e920-4b8e-ae42-e177f6cf25ca service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:12.285131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 938/1939] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:11 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:12.346119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7385b9d-51dc-40d9-969a-a3f461e45abe service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:12.354832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:12.356126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:12.358271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:12.358690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] PATH_INFO: `/v3/projects/83eb374fbdcc40bbae715ee3112d063e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:12.361230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83eb374fbdcc40bbae715ee3112d063e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:12.365097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:12.501241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83eb374fbdcc40bbae715ee3112d063e', 'request_id': 'req-410123e3-9e86-4d14-a4c9-9b660452d213'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:12.515738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83eb374fbdcc40bbae715ee3112d063e', 'request_id': 'req-410123e3-9e86-4d14-a4c9-9b660452d213'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:12.538702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83eb374fbdcc40bbae715ee3112d063e', 'request_id': 'req-410123e3-9e86-4d14-a4c9-9b660452d213'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:12.583860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] The token cache is being invalidate because project 83eb374fbdcc40bbae715ee3112d063e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:12.584580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:12.591522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-410123e3-9e86-4d14-a4c9-9b660452d213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:12.594751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1002/1940] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:12 2022] DELETE /identity/v3/projects/83eb374fbdcc40bbae715ee3112d063e => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:12.603103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:12.603683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:12.604248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:12.635356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:12.637063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:12.760601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 939/1941] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:12.779874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:12.780383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:12.780885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:12.874133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:12.876579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:12.926680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1003/1942] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:12.928101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:12.950345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb02b-d60f-4ad5-af03-71c56aabb71b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:12.962250 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:12.963308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:12.963308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:12.963523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:12.964769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:12.965085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:12.966849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:12.982138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:13.024581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 940/1943] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:13.027009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.070694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81def-fcf7-4e18-b876-d028af1c7867 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.105751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.107581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.108070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.108496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.109335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.120833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.245867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1004/1944] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:13 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:13.246155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.259189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17711a85-df1b-4087-a264-e7a8f93cb13d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.391266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.392314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.392786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.393241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.395156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.399578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.454006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 941/1945] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:13 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:13.460978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.480723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f9065-7468-4d4a-b521-6566717909a9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.490063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.491498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.492288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.492963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.494820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.498408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.525793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1005/1946] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:13.526796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.550566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d86826-82ca-41e0-b752-5b2eaefc32b5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.565819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a3713-7780-465d-82c1-f901cf71b9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.567342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a3713-7780-465d-82c1-f901cf71b9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.567884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a3713-7780-465d-82c1-f901cf71b9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.568345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a3713-7780-465d-82c1-f901cf71b9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.569234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a3713-7780-465d-82c1-f901cf71b9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.578927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a3713-7780-465d-82c1-f901cf71b9a5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.611570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 942/1947] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:13.619800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.658023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1693248-aed3-4bb9-8fb1-a555be75677f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.684502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.685952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.693602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.709776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] PATH_INFO: `/v3/projects/6697ad3966144bd7a9ea59c879445ea2/users/92935d52fc914d6f9aab8118bcb5ac36/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.750048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6697ad3966144bd7a9ea59c879445ea2, user_id=92935d52fc914d6f9aab8118bcb5ac36, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.759112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.794100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1006/1948] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:13 2022] PUT /identity/v3/projects/6697ad3966144bd7a9ea59c879445ea2/users/92935d52fc914d6f9aab8118bcb5ac36/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:13.815733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.816465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.816947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.915221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] MFA Rules not processed for user `92935d52fc914d6f9aab8118bcb5ac36`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:13.917442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:14.095656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 943/1949] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:14.111707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.114830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:14.261088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.295279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673615e-213f-4dcb-a81e-91b167f08c06 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.302662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59480389-873e-410f-9cce-88016d73c9da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.304176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480389-873e-410f-9cce-88016d73c9da service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.304654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480389-873e-410f-9cce-88016d73c9da service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.305358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59480389-873e-410f-9cce-88016d73c9da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.321643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59480389-873e-410f-9cce-88016d73c9da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.323288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59480389-873e-410f-9cce-88016d73c9da service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.339122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1007/1950] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:14.364722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.499358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90f3db-807e-437a-b485-81fdba1751fb None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.509491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.511039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.511642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.512167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] PATH_INFO: `/v3/users/92935d52fc914d6f9aab8118bcb5ac36` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.514390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92935d52fc914d6f9aab8118bcb5ac36)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.519003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.535835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.648500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92935d52fc914d6f9aab8118bcb5ac36'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:14.649337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92935d52fc914d6f9aab8118bcb5ac36'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:14.683320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92935d52fc914d6f9aab8118bcb5ac36'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:14.700072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92935d52fc914d6f9aab8118bcb5ac36'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:14.702338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92935d52fc914d6f9aab8118bcb5ac36'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:14.726372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 944/1951] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:14 2022] DELETE /identity/v3/users/92935d52fc914d6f9aab8118bcb5ac36 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:14.733486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.736235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.737039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.784375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:14.786216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:14.937720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1008/1952] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:15.009948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:15.012406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:15.232740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa380-301a-4b0b-b9bd-46d194108fa8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:15.278956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512d35e-2e10-4e21-8f27-a695c1b684fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:15.280935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512d35e-2e10-4e21-8f27-a695c1b684fa service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:15.284023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512d35e-2e10-4e21-8f27-a695c1b684fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:15.284583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512d35e-2e10-4e21-8f27-a695c1b684fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:15.335893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512d35e-2e10-4e21-8f27-a695c1b684fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:15.339740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512d35e-2e10-4e21-8f27-a695c1b684fa service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:15.359045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 945/1953] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:16.948064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:16.951578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:16.988969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e079ba-f0f7-41e0-a0a2-7de9e790a965 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.035153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:17.038815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.039399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.056498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] PATH_INFO: `/v3/projects/6697ad3966144bd7a9ea59c879445ea2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.062468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6697ad3966144bd7a9ea59c879445ea2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:17.086951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:17.328393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6697ad3966144bd7a9ea59c879445ea2', 'request_id': 'req-7e158a12-2f38-44b3-87a5-67bad095920f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:17.353484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6697ad3966144bd7a9ea59c879445ea2', 'request_id': 'req-7e158a12-2f38-44b3-87a5-67bad095920f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:17.369669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6697ad3966144bd7a9ea59c879445ea2', 'request_id': 'req-7e158a12-2f38-44b3-87a5-67bad095920f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:17.386020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] The token cache is being invalidate because project 6697ad3966144bd7a9ea59c879445ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:17.389173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:17.390448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7e158a12-2f38-44b3-87a5-67bad095920f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:17.411841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1009/1954] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:16 2022] DELETE /identity/v3/projects/6697ad3966144bd7a9ea59c879445ea2 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:17.418159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.419563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.420269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.531648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:17.538382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:17.707872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 946/1955] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:17.712953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.713460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.713743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.763549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:17.767555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:17.783822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1010/1956] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:17.785453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:17.798428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5bbcf-cfec-409c-ac48-2cb48d60ca65 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.808336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:17.809201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.809588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.810561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.811730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:17.812593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:17.814739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:17.838043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 947/1957] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:17.839549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:17.858818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35816f-30d8-4a46-96a2-11abea9b272d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.866005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:17.867486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.868079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.868712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.869903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:17.872802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:17.945286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1011/1958] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:17 2022] POST /identity/v3/projects => generated 359 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:17.947961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:17.973433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea728fd-f4d0-4c4b-bf79-c148da2851b0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.987845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:18.172869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:18.173998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:18.174618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:18.175377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:18.176715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:18.180118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:18.245332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 948/1959] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:17 2022] POST /identity/v3/users => generated 280 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:18.246661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:18.262213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8420b-067c-4355-869c-f9b780eec7ab admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:18.273077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:18.276009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:18.276305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:18.276786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:18.279094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:18.284589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:18.313521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1012/1960] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:18.317773 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:18.335242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82b51e-2fea-4854-a3e7-78b8a8aabd10 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:18.346562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e8ba1-235b-4501-adbb-1436d1dd0025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:18.347724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8ba1-235b-4501-adbb-1436d1dd0025 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:18.348123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8ba1-235b-4501-adbb-1436d1dd0025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:18.348406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e8ba1-235b-4501-adbb-1436d1dd0025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:18.349246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e8ba1-235b-4501-adbb-1436d1dd0025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:18.359896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e8ba1-235b-4501-adbb-1436d1dd0025 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:18.374492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 949/1961] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:18.376153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:18.392290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8f6a-248c-4ea2-ae59-ba8aa91f864c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:18.401184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:18.401915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:18.402198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:18.402474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] PATH_INFO: `/v3/projects/cfeb17b9039044d3b50c4c4a16878cf0/users/5f9740ca26214866a6bd550753e0b658/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:18.445505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfeb17b9039044d3b50c4c4a16878cf0, user_id=5f9740ca26214866a6bd550753e0b658, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:18.448322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:18.484499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:18.527455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1013/1962] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:18 2022] PUT /identity/v3/projects/cfeb17b9039044d3b50c4c4a16878cf0/users/5f9740ca26214866a6bd550753e0b658/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:18.530132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:18.530927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:18.531082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:18.600376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] MFA Rules not processed for user `5f9740ca26214866a6bd550753e0b658`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:18.602275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:18.768306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 950/1963] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:18.830705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:18.832695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:18.946427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcf9f3-08ce-4b97-9716-529ffb145ccb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:18.953942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4155d-0f8f-460f-b316-dfb20251d3a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:18.955972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4155d-0f8f-460f-b316-dfb20251d3a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:18.957127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4155d-0f8f-460f-b316-dfb20251d3a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:18.958501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4155d-0f8f-460f-b316-dfb20251d3a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:18.978108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4155d-0f8f-460f-b316-dfb20251d3a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:18.981010 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4155d-0f8f-460f-b316-dfb20251d3a4 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:19.018118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1014/1964] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:18 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:19.035497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:19.179663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376b87b-d5b9-47a0-beb0-f08e82f79fc4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:19.185425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:19.186118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:19.186399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:19.186673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] PATH_INFO: `/v3/users/5f9740ca26214866a6bd550753e0b658` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:19.188090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f9740ca26214866a6bd550753e0b658)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:19.192127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:19.235055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:19.294622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f9740ca26214866a6bd550753e0b658'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:19.298396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f9740ca26214866a6bd550753e0b658'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:19.323753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f9740ca26214866a6bd550753e0b658'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:19.334209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f9740ca26214866a6bd550753e0b658'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:19.335224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f9740ca26214866a6bd550753e0b658'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:19.363328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 951/1965] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:19 2022] DELETE /identity/v3/users/5f9740ca26214866a6bd550753e0b658 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:19.365495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:19.366531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:19.367002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:19.398983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:19.400610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:19.549734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1015/1966] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:19.593220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:19.595300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:19.794565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60554e8c-4b28-4401-8c85-2e7c3af78d86 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:19.799238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc500283-d34a-472b-b935-db8f6510654a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:19.799680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc500283-d34a-472b-b935-db8f6510654a service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:19.799990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc500283-d34a-472b-b935-db8f6510654a service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:19.800380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc500283-d34a-472b-b935-db8f6510654a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:19.823903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc500283-d34a-472b-b935-db8f6510654a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:19.825439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc500283-d34a-472b-b935-db8f6510654a service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:19.852204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 952/1967] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:21.164389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:21.165996 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:21.182385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30016be2-5043-426c-ba0d-cb27b4159299 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:21.193314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:21.194119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:21.199305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:21.199727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] PATH_INFO: `/v3/projects/cfeb17b9039044d3b50c4c4a16878cf0` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:21.201420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfeb17b9039044d3b50c4c4a16878cf0)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:21.203400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:21.401139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfeb17b9039044d3b50c4c4a16878cf0', 'request_id': 'req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:21.454913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfeb17b9039044d3b50c4c4a16878cf0', 'request_id': 'req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:21.482421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfeb17b9039044d3b50c4c4a16878cf0', 'request_id': 'req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:21.518920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] The token cache is being invalidate because project cfeb17b9039044d3b50c4c4a16878cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:21.519188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:21.523925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-36e25c8f-ad62-4a3a-a3e3-69fe248d7d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:21.529247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1016/1968] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:21 2022] DELETE /identity/v3/projects/cfeb17b9039044d3b50c4c4a16878cf0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:21.535988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:21.537050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:21.537626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:21.572640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:21.574380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:21.679821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 953/1969] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:21.685825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:21.686112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:21.686448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:21.718580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:21.720295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:21.735824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1017/1970] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:21.740327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:21.758847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553f809-6140-40b7-865c-19c1435ba805 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:21.766160 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:21.767278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:21.767708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:21.768207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:21.769310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:21.769828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:21.772009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:21.791021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 954/1971] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:21.797020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:21.823276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2283b-04bf-4ed9-b87a-cd13379c6c76 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:21.851311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8eff-7189-451d-aef5-f0888356adca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:21.855664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8eff-7189-451d-aef5-f0888356adca admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:21.856128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8eff-7189-451d-aef5-f0888356adca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:21.857075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8eff-7189-451d-aef5-f0888356adca admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:21.857973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8eff-7189-451d-aef5-f0888356adca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:21.865157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8eff-7189-451d-aef5-f0888356adca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:21.950400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:21.966472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1018/1972] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:21 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:21.967755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:21.991239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc347d-13bb-4db0-a6ab-5025e087448a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.120211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:22.183131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.184475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.184475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.227919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.235523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.238078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.238988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.241800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.245308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.246968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] MFA Rules not processed for user `23a95d9f48c64b0fbb8879d361bc177d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:22.249594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:22.296144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 955/1973] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:21 2022] POST /identity/v3/users => generated 289 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:22.304332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.323052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db0dc1-9323-436f-929c-047b4ee9d3ce admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.340191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.340914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.341216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.341492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.342260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.344189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.374163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 956/1974] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:22.378450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.382945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1019/1975] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:22.388418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.394572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91058ace-21c3-42f5-bf90-6412415c9a3f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.397605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:22.400866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.401674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.402031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.402375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.403170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.405622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.426204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 957/1976] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:22.428719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.464324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eb1e9-5d9b-46d6-a3ca-2707e574ddd1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.464576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0d97a-f12f-4688-a602-9f5671fad20d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.470537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.471281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69bb67-5d3e-432e-b98b-2d3f9b95a6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.471497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.472076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.472526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69bb67-5d3e-432e-b98b-2d3f9b95a6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.472676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.473100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69bb67-5d3e-432e-b98b-2d3f9b95a6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.473704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.474098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69bb67-5d3e-432e-b98b-2d3f9b95a6a0 admin admin] PATH_INFO: `/v3/projects/539c3dc46d724901aaa21dfae2c91071/users/4dfd9273258546c19ae43a611dfa47e2/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.477672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.504499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69bb67-5d3e-432e-b98b-2d3f9b95a6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539c3dc46d724901aaa21dfae2c91071, user_id=4dfd9273258546c19ae43a611dfa47e2, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.509096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69bb67-5d3e-432e-b98b-2d3f9b95a6a0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.522111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1020/1977] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:22 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:22.527441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.537969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 958/1978] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:22 2022] PUT /identity/v3/projects/539c3dc46d724901aaa21dfae2c91071/users/4dfd9273258546c19ae43a611dfa47e2/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:22.555950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e8f7-1830-450f-953f-69646449015e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.557186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e8f7-1830-450f-953f-69646449015e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.557859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e8f7-1830-450f-953f-69646449015e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.657363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aab29-9ba7-432c-8c72-8a8092ff4c53 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.704230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a443e8f7-1830-450f-953f-69646449015e None None] MFA Rules not processed for user `4dfd9273258546c19ae43a611dfa47e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:22.708480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a443e8f7-1830-450f-953f-69646449015e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:22.844604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.845274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.845564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.845831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.846531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.849133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.888496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 959/1979] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:22.896302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443e8f7-1830-450f-953f-69646449015e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.929772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1021/1980] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:22 2022] POST /identity/v3/users => generated 299 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:22.931690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443e8f7-1830-450f-953f-69646449015e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.933724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.950409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.951560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.952153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.953090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:22.954246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.957179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.962951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b283f9-9e8c-4fa8-93a7-20aac6777c09 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.974165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.975270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.975940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.977100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.989113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.994682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.997145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.051866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1022/1981] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:23.059405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.082332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 960/1982] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:22 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:23.088974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.089301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ad2df-0f94-4123-a06a-6a64234fa77e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.212004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.213071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.213454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.214121 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] PATH_INFO: `/v3/projects/e9414c57cfca4e9b9b22472e24e78b2c/users/f458111d5906467fadd22de38f32cf59/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.229936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:23.246470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9414c57cfca4e9b9b22472e24e78b2c, user_id=f458111d5906467fadd22de38f32cf59, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:23.249885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.259534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad9f15-e3dc-4f66-b566-f5c60286e1da admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.272096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.278685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.279060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.279339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.280541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:23.286136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.297181 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1023/1983] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:23 2022] PUT /identity/v3/projects/e9414c57cfca4e9b9b22472e24e78b2c/users/f458111d5906467fadd22de38f32cf59/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:23.301456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.359354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 961/1984] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:23 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:23.363690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.383625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d34c2-7eaf-465a-a14f-39cbcb791726 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.458783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.459488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.459790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.460380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.461097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:23.462925 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.475194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 962/1985] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:23.479689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.505647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611bd99-b77c-4371-813e-68ff1133d045 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.521685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.522732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.522732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.523076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.524338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:23.527577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.557175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 963/1986] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:23.563134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.585649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c36f2c-2b5c-4eab-8e1b-0317d073a72f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.595748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18c2e-f6f3-460c-a156-079b86dc6f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.597908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18c2e-f6f3-460c-a156-079b86dc6f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.599647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18c2e-f6f3-460c-a156-079b86dc6f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.600137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18c2e-f6f3-460c-a156-079b86dc6f3f admin admin] PATH_INFO: `/v3/projects/6281469952bd40839f5d3d9e59116b3b/users/cab58259a68c4aba9f04a8052ef49c39/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.619182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7cb41-89b7-48cf-b90e-ffaf9b1c0204 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.640468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.645806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.645806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.646202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.646658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:23.652849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.656338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18c2e-f6f3-460c-a156-079b86dc6f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6281469952bd40839f5d3d9e59116b3b, user_id=cab58259a68c4aba9f04a8052ef49c39, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:23.659117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18c2e-f6f3-460c-a156-079b86dc6f3f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:23.694613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1024/1987] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:23.695416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.714298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 964/1988] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:23 2022] PUT /identity/v3/projects/6281469952bd40839f5d3d9e59116b3b/users/cab58259a68c4aba9f04a8052ef49c39/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:23.726023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.726758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.727033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:23.740503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aadc0-40c1-4638-b720-920df118a4d3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:23.803115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] MFA Rules not processed for user `cab58259a68c4aba9f04a8052ef49c39`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:23.804429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:23.985496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 965/1989] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:23.993058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:23.995238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:23.996766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:23.999046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:23.999708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] PATH_INFO: `/v3/projects/e9414c57cfca4e9b9b22472e24e78b2c/users/f458111d5906467fadd22de38f32cf59/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:24.008216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9414c57cfca4e9b9b22472e24e78b2c, user_id=f458111d5906467fadd22de38f32cf59, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:24.016906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:24.023161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a8bd-54a7-4ccd-8bac-22f7cfc10072 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:24.031108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:24.036555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:24.036910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:24.037192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:24.037914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:24.044317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:24.058998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1025/1990] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:23 2022] PUT /identity/v3/projects/e9414c57cfca4e9b9b22472e24e78b2c/users/f458111d5906467fadd22de38f32cf59/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:24.101875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:24.112962 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 966/1991] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:23 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:24.117756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:24.141959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:24.385881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee0d85-d9f2-4c06-85bd-635b86f23ea5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:24.540338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:24.543014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:24.543493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:24.543934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:24.545129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:24.549850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:24.607951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1026/1992] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:24 2022] POST /identity/v3/users => generated 295 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:24.612797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:24.646901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b6b0d-bfb4-44ba-b2b4-b0ff95e6124a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:24.672450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:24.679826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:24.680428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:24.680995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:24.682363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:24.685637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:24.727906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:24.728540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 967/1993] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:24.748426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572e96e-b76f-4be6-859c-6f3281fc3388 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:24.761342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:24.762705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:24.762705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:24.763025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] PATH_INFO: `/v3/projects/5004488c19674d5f8498330bdddc398a/users/70acb4e13b76438190fb12f2868353a6/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:24.816415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5004488c19674d5f8498330bdddc398a, user_id=70acb4e13b76438190fb12f2868353a6, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:24.825251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:24.858877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1027/1994] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:24 2022] PUT /identity/v3/projects/5004488c19674d5f8498330bdddc398a/users/70acb4e13b76438190fb12f2868353a6/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:24.864228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:24.880630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d4771-7f2c-423d-9007-3011c3bcbb5d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:25.031206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c307d-533b-4b4b-a159-323632e6b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:25.031698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c307d-533b-4b4b-a159-323632e6b5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.032658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c307d-533b-4b4b-a159-323632e6b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.032658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c307d-533b-4b4b-a159-323632e6b5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.033324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c307d-533b-4b4b-a159-323632e6b5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:25.035358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c307d-533b-4b4b-a159-323632e6b5f9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:25.073120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 968/1995] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:25.074305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:25.092138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d861043-d1d4-4823-8289-a4a7f83cd9d7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:25.110263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:25.110594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.110729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.111194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] PATH_INFO: `/v3/projects/5004488c19674d5f8498330bdddc398a/users/70acb4e13b76438190fb12f2868353a6/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.117806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5004488c19674d5f8498330bdddc398a, user_id=70acb4e13b76438190fb12f2868353a6, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:25.129908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:25.202475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1028/1996] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:25 2022] PUT /identity/v3/projects/5004488c19674d5f8498330bdddc398a/users/70acb4e13b76438190fb12f2868353a6/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:25.302382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.302779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.303118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.344169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] MFA Rules not processed for user `70acb4e13b76438190fb12f2868353a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:25.345969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:25.554846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 969/1997] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:25.576117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:25.578332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:25.716755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6d29e-d042-49e6-8301-30954116b7b4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:25.727951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:25.728619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.728899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.729176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.790345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:25.876566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:25.880320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:25.900599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1029/1998] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:28.021128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:28.057514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221253ed-5500-4e2e-8f59-58d12907635a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:28.066327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:28.067173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:28.067731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:28.068367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:28.274466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:28.276635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:28.305891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 970/1999] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:29.533829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:29.546653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f2c8-6df7-47b4-8c3e-9fefc7126097 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:29.551384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6363-1216-4d4e-9cc2-64159d09ad8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:29.552579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6363-1216-4d4e-9cc2-64159d09ad8a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:29.553162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6363-1216-4d4e-9cc2-64159d09ad8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:29.553626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6363-1216-4d4e-9cc2-64159d09ad8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:29.569691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6363-1216-4d4e-9cc2-64159d09ad8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:29.571206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6363-1216-4d4e-9cc2-64159d09ad8a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:29.586076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1030/2000] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:32.999416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:33.001749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:33.038185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:33.152771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b061a5-e0be-4b41-bc1f-dd8bbe343de5 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:33.157793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:33.158627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:33.159009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:33.159288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] PATH_INFO: `/v3/users/d5b413c7ba0d4836b3661b91b6431370` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:33.160690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b413c7ba0d4836b3661b91b6431370)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:33.163048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:33.231033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b413c7ba0d4836b3661b91b6431370'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:33.232701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b413c7ba0d4836b3661b91b6431370'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:33.272638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b413c7ba0d4836b3661b91b6431370'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:33.295846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b413c7ba0d4836b3661b91b6431370'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:33.296644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b413c7ba0d4836b3661b91b6431370'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:33.335783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 971/2001] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:32 2022] DELETE /identity/v3/users/d5b413c7ba0d4836b3661b91b6431370 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:33.338460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0af23e-5256-4034-893f-2f48c41f312c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:33.339208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0af23e-5256-4034-893f-2f48c41f312c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:33.339377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0af23e-5256-4034-893f-2f48c41f312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:33.378826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cb0af23e-5256-4034-893f-2f48c41f312c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:33.380197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cb0af23e-5256-4034-893f-2f48c41f312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:33.540294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1031/2002] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:33.558645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:33.560880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:33.666265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbd5fd-c0a6-440f-b146-66c78f8d569c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:33.678644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:33.681031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:33.681375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:33.681807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:33.701549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:33.706080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:33.729847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 972/2003] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:33.745039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442fa7a-9c2c-4829-8388-54ab4d0657a4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:33.746172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442fa7a-9c2c-4829-8388-54ab4d0657a4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:33.746845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442fa7a-9c2c-4829-8388-54ab4d0657a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:33.792297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5442fa7a-9c2c-4829-8388-54ab4d0657a4 None None] MFA Rules not processed for user `f458111d5906467fadd22de38f32cf59`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:33.793942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5442fa7a-9c2c-4829-8388-54ab4d0657a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:33.914639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1032/2004] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:33.929993 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:33.931460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:33.975059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:34.031420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8d67b-51dc-4814-94af-aac4f8aecaed service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:34.035725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:34.036383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:34.036660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:34.036924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:34.038789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:34.130357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:34.132180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:34.144829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 973/2005] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:34.394186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:34.400381 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:34.401032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:34.443005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:34.445091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:34.496611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:34.550295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1033/2006] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:40:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:34.648046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:34.649116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:34.649764 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:34.651229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:34.699258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224710c-4c1c-4899-b2aa-a66e4704b35f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:34.703906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:34.704783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:34.705057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:34.705476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] PATH_INFO: `/v3/projects/af0c16f98948427caaf8d4f1c2e909e7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:34.707011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0c16f98948427caaf8d4f1c2e909e7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:34.709173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:34.760899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0c16f98948427caaf8d4f1c2e909e7', 'request_id': 'req-22a69a90-602c-4c76-9ac5-aae02399f4f2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:34.768612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0c16f98948427caaf8d4f1c2e909e7', 'request_id': 'req-22a69a90-602c-4c76-9ac5-aae02399f4f2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:34.792390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0c16f98948427caaf8d4f1c2e909e7', 'request_id': 'req-22a69a90-602c-4c76-9ac5-aae02399f4f2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:34.795244 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf74ab1-7f02-473a-8856-6c96c57e748e service glance] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:34.810972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] The token cache is being invalidate because project af0c16f98948427caaf8d4f1c2e909e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:34.811267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:34.811991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:34.813619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1034/2007] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:34 2022] DELETE /identity/v3/projects/af0c16f98948427caaf8d4f1c2e909e7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:34.819711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:34.821859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:34.942847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:34.943967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:34.944404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:34.944840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:34.948026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:34.960286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a69a90-602c-4c76-9ac5-aae02399f4f2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:34.964610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:34.967236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:34.968139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:34.968853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:34.969282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] PATH_INFO: `/v3/users/d1036c61174948a58bfbf3b1d8fa7b0d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:34.970907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1036c61174948a58bfbf3b1d8fa7b0d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:34.973730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:35.060149 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1036c61174948a58bfbf3b1d8fa7b0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.061104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1036c61174948a58bfbf3b1d8fa7b0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.071289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:35.073627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:35.078378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1036c61174948a58bfbf3b1d8fa7b0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.086970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 974/2008] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:40:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:40:35.088141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:35.089636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:35.105571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:35.132212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1036c61174948a58bfbf3b1d8fa7b0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.166655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3492c070-4b5f-4560-b156-3fd06f0bc599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1036c61174948a58bfbf3b1d8fa7b0d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.169165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0c0c8-3c7a-428b-bf48-fde7339dfbbd service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:35.177110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1035/2009] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:34 2022] DELETE /identity/v3/users/d1036c61174948a58bfbf3b1d8fa7b0d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:35.217012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:35.218067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:35.218512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:35.218907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] PATH_INFO: `/v3/users/4dfd9273258546c19ae43a611dfa47e2` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:35.228657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfd9273258546c19ae43a611dfa47e2)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:35.233202 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:35.475049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfd9273258546c19ae43a611dfa47e2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.493964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfd9273258546c19ae43a611dfa47e2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.550168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfd9273258546c19ae43a611dfa47e2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.582350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfd9273258546c19ae43a611dfa47e2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.589026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfd9273258546c19ae43a611dfa47e2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:35.649159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 975/2010] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:35 2022] DELETE /identity/v3/users/4dfd9273258546c19ae43a611dfa47e2 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:35.655931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:35.656694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:35.656817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:35.761843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:35.768512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:36.054466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1036/2011] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:36.092818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:36.095258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:36.316395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:36.318274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:36.355156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020f9b1-d75b-4e2c-995f-a9ffaa345870 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:36.359242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caff66e-482d-4ad3-97e8-3c2d9ec7d69e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:36.367520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55545a-560c-4ac5-9e69-7fe75637fbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:36.368361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55545a-560c-4ac5-9e69-7fe75637fbbc service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:36.368658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55545a-560c-4ac5-9e69-7fe75637fbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:36.368931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55545a-560c-4ac5-9e69-7fe75637fbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:36.393897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:36.402066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:36.403453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:36.404884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55545a-560c-4ac5-9e69-7fe75637fbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:36.405482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] PATH_INFO: `/v3/projects/980c4a34c2ca4484bf806fd9abcd72fd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:36.410294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55545a-560c-4ac5-9e69-7fe75637fbbc service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:36.416103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980c4a34c2ca4484bf806fd9abcd72fd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:36.433939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:36.478397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 976/2012] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:36.748978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980c4a34c2ca4484bf806fd9abcd72fd', 'request_id': 'req-1626e954-c7ed-46eb-877a-4d6d44dda5c6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:36.805210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980c4a34c2ca4484bf806fd9abcd72fd', 'request_id': 'req-1626e954-c7ed-46eb-877a-4d6d44dda5c6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:36.827780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980c4a34c2ca4484bf806fd9abcd72fd', 'request_id': 'req-1626e954-c7ed-46eb-877a-4d6d44dda5c6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:36.866009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] The token cache is being invalidate because project 980c4a34c2ca4484bf806fd9abcd72fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:36.867736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:36.869341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1626e954-c7ed-46eb-877a-4d6d44dda5c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:36.873543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1037/2013] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:36 2022] DELETE /identity/v3/projects/980c4a34c2ca4484bf806fd9abcd72fd => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:36.880243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:36.881505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:36.881985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:36.921402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:36.922971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:36.965227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:37.106069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 977/2014] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:37.107799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.108085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.108364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.153039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:37.154502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:37.157355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:37.160508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:37.178050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1038/2015] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:37.179380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:37.184715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170799a3-4aa3-4f88-91f9-55203865527d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:37.194186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:37.194862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.195140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.195414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] PATH_INFO: `/v3/projects/539c3dc46d724901aaa21dfae2c91071` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.196703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] RBAC: Authorizing `identity:delete_project(project_id=539c3dc46d724901aaa21dfae2c91071)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:37.197854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240a22-7e5e-46c9-bf2a-564dfe640a05 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:37.200977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:37.206349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:37.207249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.207249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.207560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.208255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:37.208556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:37.210202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:37.240625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1039/2016] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:40:37.242387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:37.254916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8e906-3452-4adc-9e78-6b402a70e16c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:37.262238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:37.263027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.263407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.263850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.265079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:37.270456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:37.371157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539c3dc46d724901aaa21dfae2c91071', 'request_id': 'req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.385567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539c3dc46d724901aaa21dfae2c91071', 'request_id': 'req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.386197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1040/2017] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:37 2022] POST /identity/v3/projects => generated 361 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:37.391058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:37.416476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539c3dc46d724901aaa21dfae2c91071', 'request_id': 'req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.468380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:37.478444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] The token cache is being invalidate because project 539c3dc46d724901aaa21dfae2c91071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:37.478941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.480237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.481996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 978/2018] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:37 2022] DELETE /identity/v3/projects/539c3dc46d724901aaa21dfae2c91071 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:37.486635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:37.488860 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:37.662752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce6d0a-2608-46a7-ba26-8a0119ae80db admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:37.673017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf6b30-da24-4d2b-bad2-c9939ba83dd5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:37.675705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:37.676472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.676778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.677107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] PATH_INFO: `/v3/users/cab58259a68c4aba9f04a8052ef49c39` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.679947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab58259a68c4aba9f04a8052ef49c39)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:37.681904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:37.688558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:37.689204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.689477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.689747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.690547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:37.692581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:37.759992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1041/2019] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:37 2022] POST /identity/v3/users => generated 281 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:37.772484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:37.781778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:37.816174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab58259a68c4aba9f04a8052ef49c39'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.816174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab58259a68c4aba9f04a8052ef49c39'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.821719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:37.832112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f47c4c-0935-453f-b9d7-14b9431d835d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:37.840737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab58259a68c4aba9f04a8052ef49c39'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.861899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:37.911311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:37.913688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:37.950291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:37.954960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:37.964313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:37.967942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab58259a68c4aba9f04a8052ef49c39'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.968844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab58259a68c4aba9f04a8052ef49c39'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:37.999941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 979/2020] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:37 2022] DELETE /identity/v3/users/cab58259a68c4aba9f04a8052ef49c39 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:38.018829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1042/2021] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:38.023673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:38.026337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:38.129005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f05fa7-104c-4884-8cab-91a7e2d5386c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:38.135154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ee900-0bc4-408d-9055-fa357ab7656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:38.135818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee900-0bc4-408d-9055-fa357ab7656c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:38.136297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee900-0bc4-408d-9055-fa357ab7656c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:38.136646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee900-0bc4-408d-9055-fa357ab7656c admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:38.137658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ee900-0bc4-408d-9055-fa357ab7656c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:38.139618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ee900-0bc4-408d-9055-fa357ab7656c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:38.161727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 980/2022] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:38.164306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:38.187209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f11a7-3133-4854-b49b-6fb92f4eb040 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:38.194557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:38.195314 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:38.195589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:38.195897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] PATH_INFO: `/v3/projects/b39aa2618f634a3d8e9a98703f8da477/users/122867e95b9a4e60861354f750ad01d7/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:38.244406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b39aa2618f634a3d8e9a98703f8da477, user_id=122867e95b9a4e60861354f750ad01d7, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:38.249118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:38.279335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1043/2023] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:38 2022] PUT /identity/v3/projects/b39aa2618f634a3d8e9a98703f8da477/users/122867e95b9a4e60861354f750ad01d7/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:38.292700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:38.293013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:38.293438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:38.338344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] MFA Rules not processed for user `122867e95b9a4e60861354f750ad01d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:38.339650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:38.449102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:38.451404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 981/2024] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:38.632295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb49f2d-e18b-4b3d-8c17-ee8478510f48 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:38.647217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:38.647915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:38.648198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:38.648473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:38.649177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:38.651191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:38.728158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1044/2025] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:38 2022] POST /identity/v3/projects => generated 363 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:38.730931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:38.778219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4c38a-34a9-4952-8e10-9c21c688c990 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:38.800427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:38.803049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:38.960505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:38.983639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fe604-7f8c-4e99-8605-44805537976a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:38.990574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:38.991369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:38.991853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:38.992316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:38.993045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:38.993696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:38.993974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:38.994262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] PATH_INFO: `/v3/projects/6281469952bd40839f5d3d9e59116b3b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:38.994720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:38.995576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6281469952bd40839f5d3d9e59116b3b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:38.997325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:38.997793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:39.000125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:39.051387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 982/2026] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:38 2022] POST /identity/v3/users => generated 288 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:39.063646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:39.091658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac9123-2855-43c4-aeb7-c6c8e1957b98 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:39.097951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:39.101813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:39.102374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:39.102852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:39.103941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:39.106220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:39.146421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 983/2027] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:39.150914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:39.165401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6281469952bd40839f5d3d9e59116b3b', 'request_id': 'req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.178219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb588bf-4420-4842-94c8-dd33a0c5dc5f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:39.181179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6281469952bd40839f5d3d9e59116b3b', 'request_id': 'req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.203893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6281469952bd40839f5d3d9e59116b3b', 'request_id': 'req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.225444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] The token cache is being invalidate because project 6281469952bd40839f5d3d9e59116b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:39.226116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.227829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.232749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1045/2028] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:38 2022] DELETE /identity/v3/projects/6281469952bd40839f5d3d9e59116b3b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:39.242193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:39.245622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:39.414554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:39.417983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbac4e-d9c3-46e9-bc8b-0863cce13d9e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:39.419085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:39.422894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:39.423661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] PATH_INFO: `/v3/projects/ce46d9fa1a304210b1b795be0356a7dd/users/8ea2b917802246b58423e1d051af14db/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:39.440669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:39.441853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:39.442174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:39.442559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] PATH_INFO: `/v3/users/70acb4e13b76438190fb12f2868353a6` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:39.444540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70acb4e13b76438190fb12f2868353a6)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:39.447120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:39.463887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce46d9fa1a304210b1b795be0356a7dd, user_id=8ea2b917802246b58423e1d051af14db, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:39.466821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:39.487223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 984/2029] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:39 2022] PUT /identity/v3/projects/ce46d9fa1a304210b1b795be0356a7dd/users/8ea2b917802246b58423e1d051af14db/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:39.492743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:39.495139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:39.648705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70acb4e13b76438190fb12f2868353a6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.651625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70acb4e13b76438190fb12f2868353a6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.689772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70acb4e13b76438190fb12f2868353a6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.728230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4329c-1d2a-462c-919f-01bd2577032a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:39.737699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70acb4e13b76438190fb12f2868353a6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.754051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2df4-352e-4475-b331-140bc6239106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:39.757827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2df4-352e-4475-b331-140bc6239106 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:39.760065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2df4-352e-4475-b331-140bc6239106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:39.760706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2df4-352e-4475-b331-140bc6239106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:39.761915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2df4-352e-4475-b331-140bc6239106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:39.773385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2df4-352e-4475-b331-140bc6239106 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:39.782963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70acb4e13b76438190fb12f2868353a6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:39.811551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1046/2030] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:39 2022] DELETE /identity/v3/users/70acb4e13b76438190fb12f2868353a6 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:39.828201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 985/2031] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:39.828733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:39.834653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:39.915985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:40.142488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97f31b-f470-4bac-b05e-a232b8443de1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:40.159166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:40.160211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.160490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.160762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] PATH_INFO: `/v3/projects/ce46d9fa1a304210b1b795be0356a7dd/users/8ea2b917802246b58423e1d051af14db/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.170262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce46d9fa1a304210b1b795be0356a7dd, user_id=8ea2b917802246b58423e1d051af14db, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:40.179790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:40.196654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1047/2032] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:39 2022] PUT /identity/v3/projects/ce46d9fa1a304210b1b795be0356a7dd/users/8ea2b917802246b58423e1d051af14db/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:40.207883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a00c2-d302-4494-ba98-4cd8cffc3376 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.209632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a00c2-d302-4494-ba98-4cd8cffc3376 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.210239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a00c2-d302-4494-ba98-4cd8cffc3376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.288224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c71a00c2-d302-4494-ba98-4cd8cffc3376 None None] MFA Rules not processed for user `8ea2b917802246b58423e1d051af14db`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:40.292700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c71a00c2-d302-4494-ba98-4cd8cffc3376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:40.407979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:40.467928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 986/2033] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:40.481832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:40.485079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:40.566314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be7a7e-238f-489f-94a8-1351b1f812a5 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.566597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be7a7e-238f-489f-94a8-1351b1f812a5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.566963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be7a7e-238f-489f-94a8-1351b1f812a5 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.569594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 987/2034] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:40:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:40.613628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63242-0551-4293-91f9-1bf3e888d0b5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:40.620494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:40.621174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.621462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.621783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.634881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.636662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.636968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.655548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:40.656042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:40.675818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:40.676145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:40.690772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1048/2035] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:40 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:40.720610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:40.724758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:40.878362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf711a1-b4ac-4e5c-aa28-15addf96676d service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:40.897467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:40.905871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 988/2036] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:40.906890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.906890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.910567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:40.914782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:40.914986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] PATH_INFO: `/v3/users/7a310f36f74e4eaabd9f32220feb274a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.916877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a310f36f74e4eaabd9f32220feb274a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:40.918741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:40.949041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790a47-0767-4a3a-88c1-e3b1986f08d0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:40.969098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:40.970907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:40.975438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:40.976637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] PATH_INFO: `/v3/projects/5004488c19674d5f8498330bdddc398a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:40.982203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5004488c19674d5f8498330bdddc398a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:40.986042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:41.089426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a310f36f74e4eaabd9f32220feb274a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.090225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a310f36f74e4eaabd9f32220feb274a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.094242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:41.107885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a310f36f74e4eaabd9f32220feb274a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.108760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5004488c19674d5f8498330bdddc398a', 'request_id': 'req-5ecb7022-53e8-484a-8ffe-b6d9569d48db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.118753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5004488c19674d5f8498330bdddc398a', 'request_id': 'req-5ecb7022-53e8-484a-8ffe-b6d9569d48db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.131954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5004488c19674d5f8498330bdddc398a', 'request_id': 'req-5ecb7022-53e8-484a-8ffe-b6d9569d48db'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.147523 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a310f36f74e4eaabd9f32220feb274a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.159990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] The token cache is being invalidate because project 5004488c19674d5f8498330bdddc398a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:41.160788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.163800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.165116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a310f36f74e4eaabd9f32220feb274a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:41.168466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 989/2037] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:40 2022] DELETE /identity/v3/projects/5004488c19674d5f8498330bdddc398a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:41.170379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:41.173502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.189445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1049/2038] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:40 2022] DELETE /identity/v3/users/7a310f36f74e4eaabd9f32220feb274a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:41.192757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:41.197764 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.287335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb7022-53e8-484a-8ffe-b6d9569d48db admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:41.299676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:41.324389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d59a85-aacc-4f77-b71a-49585535fdc5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:41.339204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:41.340167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.340634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.341055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.343245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.358939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:41.362340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa328f5-c394-4f8c-97a1-d8a9ff2cdc9d service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:41.382086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1050/2039] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:41.386644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.387221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.388257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.395535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:41.396328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.396742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.397157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.400126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.440155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:41.446048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.525414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:41.529000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc234ea-1bd5-4196-a715-5a01589ccddc service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:41.553402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 990/2040] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:41.597743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1051/2041] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:41.612065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.612743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.613204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.683859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:41.683859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.686671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:41.688201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:41.696777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 991/2042] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:41.701470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:41.718719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8105b7-2503-4e45-b945-29f5d1931edf None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:41.719099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525412-4bf6-4a3b-92cb-742d960fd17f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:41.729647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:41.730653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.730922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.731193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] PATH_INFO: `/v3/projects/c6d1d41a90df4eb8ae7b9a3934629614` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.737044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:41.737990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.738279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.738578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.739297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:41.739611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:41.741343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:41.788832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 992/2043] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:41.794189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:41.796128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d1d41a90df4eb8ae7b9a3934629614)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:41.837026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:41.841388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2b0a4-dd2c-4e16-a064-db623cf54cfb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:41.864363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:41.864363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:41.864363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:41.864363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:41.865413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:41.872389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:41.949197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 993/2044] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:41 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:41.954019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:41.999507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d1d41a90df4eb8ae7b9a3934629614', 'request_id': 'req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:42.002691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=1,keystone:DELETE=7,keystone:SELECT=92 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:42.012662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d1d41a90df4eb8ae7b9a3934629614', 'request_id': 'req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:42.029231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d1d41a90df4eb8ae7b9a3934629614', 'request_id': 'req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:42.054129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] The token cache is being invalidate because project c6d1d41a90df4eb8ae7b9a3934629614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:42.054876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:42.055963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2c2ec84b-e5e0-42fc-903c-9fc3765956b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:42.058781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1052/2045] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:41 2022] DELETE /identity/v3/projects/c6d1d41a90df4eb8ae7b9a3934629614 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:42.062075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfe43f-11af-42e6-bd37-27d760748e65 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.063790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8c12-ba04-4c89-bda9-72a443a1a44a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.064164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8c12-ba04-4c89-bda9-72a443a1a44a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.064557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e8c12-ba04-4c89-bda9-72a443a1a44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.092474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-095e8c12-ba04-4c89-bda9-72a443a1a44a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:42.094151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-095e8c12-ba04-4c89-bda9-72a443a1a44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:42.158947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:42.171981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1053/2046] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:42.181718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.182154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.182537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.206960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:42.208549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:42.219235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1054/2047] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:42.226202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.244013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e696373-03ae-49b7-a68c-b6bda97cdd11 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.254789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.255319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.255602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.256107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.256385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.256715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.257008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.257197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.257310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.258586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:42.258889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.259341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.261327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.301505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 994/2048] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:41 2022] POST /identity/v3/users => generated 287 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:42.303666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.305232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1055/2049] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:40:42.308054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.308572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:42.330672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e6782-c61d-4f83-a250-3015962e195d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.338898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bcd47-fc1d-4e87-9398-cd309fe6fed6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.339131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.341545 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.342887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.343479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.345156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.352321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.365180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.365845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.366126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.366427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.367479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.369589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.374892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 995/2050] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:42.379014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.395844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be1b5-25da-470d-88ee-bbf74e12b5f0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.413915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.415391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.417571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.418447 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.421707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.426237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.467217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1056/2051] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:42 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:42.468667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.469973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 996/2052] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:42.472883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.495337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f9b1c-7629-4631-8a98-d55a013b3b66 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.600338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93645d-7f9d-445c-8477-516f93809e17 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.605976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.607163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.607641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.608105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.608907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.611699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.642552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1057/2053] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:42 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:42.647173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.660305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f2577-89fe-4828-b484-e98261f6d9f5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.668117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.669747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.669747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.671802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.671802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.678448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.702353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216504d-e593-462c-ae10-ab30af51a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.703050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216504d-e593-462c-ae10-ab30af51a60b admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.703330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216504d-e593-462c-ae10-ab30af51a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.703606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216504d-e593-462c-ae10-ab30af51a60b admin admin] PATH_INFO: `/v3/projects/c31add21e55d4704b8285a700af04410/users/c4c08c97c2fb446ba2d363c8a18d4958/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.726860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1058/2054] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:42.732449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.748917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216504d-e593-462c-ae10-ab30af51a60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31add21e55d4704b8285a700af04410, user_id=c4c08c97c2fb446ba2d363c8a18d4958, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.755893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216504d-e593-462c-ae10-ab30af51a60b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.769971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153fd53-2b1c-4ea7-9571-182a391279ba admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.779409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 997/2055] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:42 2022] PUT /identity/v3/projects/c31add21e55d4704b8285a700af04410/users/c4c08c97c2fb446ba2d363c8a18d4958/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:42.792316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.792915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.793328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.806200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:42.855675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] MFA Rules not processed for user `c4c08c97c2fb446ba2d363c8a18d4958`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:42.857251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:42.896448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.897230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.897514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.897869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.899321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.903340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.918866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1059/2056] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:42.924620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:42.940879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e679a-00d0-4e07-9ab4-e3547bcc821a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:42.948311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6fc2e-0798-4843-a4ab-1a3dfca36b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:42.949141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6fc2e-0798-4843-a4ab-1a3dfca36b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:42.949367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6fc2e-0798-4843-a4ab-1a3dfca36b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:42.950775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6fc2e-0798-4843-a4ab-1a3dfca36b24 admin admin] PATH_INFO: `/v3/projects/defbddb8e99941dfb7c7a3f87806a4e6/users/e9763b8abdd74462b74bdb771bc1a7be/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:42.976187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6fc2e-0798-4843-a4ab-1a3dfca36b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=defbddb8e99941dfb7c7a3f87806a4e6, user_id=e9763b8abdd74462b74bdb771bc1a7be, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:42.983983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6fc2e-0798-4843-a4ab-1a3dfca36b24 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:42.991593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 998/2057] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:43.004627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:43.009007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:43.031829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1060/2058] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:42 2022] PUT /identity/v3/projects/defbddb8e99941dfb7c7a3f87806a4e6/users/e9763b8abdd74462b74bdb771bc1a7be/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:43.049975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:43.050327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:43.050517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:43.054293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:43.126371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] MFA Rules not processed for user `e9763b8abdd74462b74bdb771bc1a7be`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:43.128085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:43.146325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13405e3-7315-468e-b76e-fd055d6de580 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:43.242668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:43.244609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:43.244884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:43.245161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:43.291698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1061/2059] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:43.294882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:43.426483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:43.429849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:43.436446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5defa3-bbb3-468b-bf6e-56f811b90fef None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:43.450909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:43.452183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:43.452667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:43.453191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:43.454027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:43.457394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:43.460406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 999/2060] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:43.530070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1062/2061] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:43 2022] POST /identity/v3/projects => generated 369 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:43.534831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:43.559138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4728f-d75a-4090-83e0-ac25de2b4d3d service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:43.781018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:43.782017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:43.782506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:43.783005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:43.784075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:43.790749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:43.874911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1000/2062] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:43 2022] POST /identity/v3/users => generated 291 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:43.876540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:43.888585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:43.904124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e295c-6546-4c32-b852-e4b12b0d128a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:43.943336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:43.944198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:43.944483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:43.944758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:43.945631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:43.950900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:44.003452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1063/2063] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:44.013161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:44.111648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0b6e4-5772-4832-8d45-41b962470ea9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:44.135830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:44.136660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:44.136937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:44.137212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] PATH_INFO: `/v3/projects/4c7ea08516cf4ca1801d1f1f82804b0a/users/aa3034d32b5f4130bbb2efcbd03a1c37/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:44.198070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:44.226281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7ea08516cf4ca1801d1f1f82804b0a, user_id=aa3034d32b5f4130bbb2efcbd03a1c37, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:44.229581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:44.273074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1001/2064] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:44 2022] PUT /identity/v3/projects/4c7ea08516cf4ca1801d1f1f82804b0a/users/aa3034d32b5f4130bbb2efcbd03a1c37/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:44.277434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:44.305668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b74da0-5f50-489f-adcb-2b56870dcb04 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:44.459750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb26d0-2dad-49cc-b2ce-b9c5c70ba6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:44.460491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb26d0-2dad-49cc-b2ce-b9c5c70ba6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:44.460773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb26d0-2dad-49cc-b2ce-b9c5c70ba6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:44.461045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb26d0-2dad-49cc-b2ce-b9c5c70ba6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:44.461767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb26d0-2dad-49cc-b2ce-b9c5c70ba6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:44.463546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb26d0-2dad-49cc-b2ce-b9c5c70ba6d9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:44.479915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1064/2065] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:44.482083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:44.503411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a390-c6ff-4c55-a451-70f78b153411 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:44.517179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:44.518028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:44.518501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:44.518986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] PATH_INFO: `/v3/projects/4c7ea08516cf4ca1801d1f1f82804b0a/users/aa3034d32b5f4130bbb2efcbd03a1c37/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:44.522790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7ea08516cf4ca1801d1f1f82804b0a, user_id=aa3034d32b5f4130bbb2efcbd03a1c37, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:44.531401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:44.551827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1002/2066] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:44 2022] PUT /identity/v3/projects/4c7ea08516cf4ca1801d1f1f82804b0a/users/aa3034d32b5f4130bbb2efcbd03a1c37/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:44.570578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:44.571055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:44.571551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:44.620140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] MFA Rules not processed for user `aa3034d32b5f4130bbb2efcbd03a1c37`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:44.621401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:44.758867 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1065/2067] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:44.772286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:44.861235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258ccee-8531-4ee4-8ef5-2da3c3b5c7e6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:44.868872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a153d21-e1be-40f6-82d4-3c9ab10057a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:44.869998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153d21-e1be-40f6-82d4-3c9ab10057a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:44.870794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153d21-e1be-40f6-82d4-3c9ab10057a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:44.871359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153d21-e1be-40f6-82d4-3c9ab10057a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:44.888053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a153d21-e1be-40f6-82d4-3c9ab10057a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:44.889669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a153d21-e1be-40f6-82d4-3c9ab10057a1 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:44.905265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1003/2068] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:44.974428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:44.976087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:45.000053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728b7b3-0388-4e7c-8dca-7b59758494d0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:45.006044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:45.007269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:45.007816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:45.008670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:45.110504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:45.112667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:45.125284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1066/2069] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:45.738921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591e1c-d78c-4a69-acd2-c31d81786123 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:45.739912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591e1c-d78c-4a69-acd2-c31d81786123 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:45.740232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591e1c-d78c-4a69-acd2-c31d81786123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:45.784028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d6591e1c-d78c-4a69-acd2-c31d81786123 None None] MFA Rules not processed for user `c4c08c97c2fb446ba2d363c8a18d4958`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:45.785362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6591e1c-d78c-4a69-acd2-c31d81786123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:45.882433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1004/2070] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:49.301325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:49.319075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:49.392120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5bb9d-7545-41d6-a9bd-8a0ccd96083c service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:49.397269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:49.397944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:49.398223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:49.398497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] PATH_INFO: `/v3/users/e9763b8abdd74462b74bdb771bc1a7be` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:49.399713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9763b8abdd74462b74bdb771bc1a7be)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:49.401944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:49.465588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9763b8abdd74462b74bdb771bc1a7be'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:49.466441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9763b8abdd74462b74bdb771bc1a7be'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:49.487643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9763b8abdd74462b74bdb771bc1a7be'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:49.512211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9763b8abdd74462b74bdb771bc1a7be'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:49.523747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9763b8abdd74462b74bdb771bc1a7be'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:49.553720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1067/2071] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:49 2022] DELETE /identity/v3/users/e9763b8abdd74462b74bdb771bc1a7be => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:49.561576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:49.562162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:49.562162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:49.585288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:49.586617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:49.684668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1005/2072] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:49.695838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:49.697467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:49.784359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a85ec-d0c8-41af-828d-798e6c76ec87 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:49.790081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:49.791077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:49.791456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:49.792139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:49.807750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:49.810042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:49.824268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1068/2073] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:50.255224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:50.257155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:50.278162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2277a1-5225-4ac4-87bb-a314e323e33a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:50.295526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:50.296442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:50.296622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:50.296904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] PATH_INFO: `/v3/projects/defbddb8e99941dfb7c7a3f87806a4e6` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:50.299566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] RBAC: Authorizing `identity:delete_project(project_id=defbddb8e99941dfb7c7a3f87806a4e6)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:50.301876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:50.343677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:50.407487 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'defbddb8e99941dfb7c7a3f87806a4e6', 'request_id': 'req-b016d196-9dfd-462f-a8cb-15f060dc862d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.417967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'defbddb8e99941dfb7c7a3f87806a4e6', 'request_id': 'req-b016d196-9dfd-462f-a8cb-15f060dc862d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.432518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'defbddb8e99941dfb7c7a3f87806a4e6', 'request_id': 'req-b016d196-9dfd-462f-a8cb-15f060dc862d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.448980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] The token cache is being invalidate because project defbddb8e99941dfb7c7a3f87806a4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:50.449687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.450506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b016d196-9dfd-462f-a8cb-15f060dc862d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.452107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1006/2074] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:50 2022] DELETE /identity/v3/projects/defbddb8e99941dfb7c7a3f87806a4e6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:50.456384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:50.474670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ced45-f419-4c35-89b3-0999231a193b service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:50.542722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:50.589409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:50.590442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:50.590442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:50.590942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] PATH_INFO: `/v3/users/aa3034d32b5f4130bbb2efcbd03a1c37` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:50.592469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3034d32b5f4130bbb2efcbd03a1c37)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:50.595646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:50.655201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3034d32b5f4130bbb2efcbd03a1c37'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.656005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3034d32b5f4130bbb2efcbd03a1c37'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.666605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3034d32b5f4130bbb2efcbd03a1c37'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.685692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3034d32b5f4130bbb2efcbd03a1c37'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.695005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3034d32b5f4130bbb2efcbd03a1c37'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:50.703902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1069/2075] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:50 2022] DELETE /identity/v3/users/aa3034d32b5f4130bbb2efcbd03a1c37 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:50.788707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbb7a4-9ceb-434f-8a01-ef146fc6980c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:50.789410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbb7a4-9ceb-434f-8a01-ef146fc6980c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:50.789410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbb7a4-9ceb-434f-8a01-ef146fc6980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:50.811471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-87cbb7a4-9ceb-434f-8a01-ef146fc6980c None None] MFA Rules not processed for user `23a95d9f48c64b0fbb8879d361bc177d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:50.813037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-87cbb7a4-9ceb-434f-8a01-ef146fc6980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:50.904417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1007/2076] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:51.053716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.056483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:51.142189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879b584-73b4-44f0-bde3-2a53d361d7d0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.149161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.149888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.150341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.150736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] PATH_INFO: `/v3/projects/4c7ea08516cf4ca1801d1f1f82804b0a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.152060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7ea08516cf4ca1801d1f1f82804b0a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.154015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.226412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7ea08516cf4ca1801d1f1f82804b0a', 'request_id': 'req-c5f1e7eb-460e-4792-8e15-05c7e517629c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:51.232385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:51.236642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7ea08516cf4ca1801d1f1f82804b0a', 'request_id': 'req-c5f1e7eb-460e-4792-8e15-05c7e517629c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:51.254906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7ea08516cf4ca1801d1f1f82804b0a', 'request_id': 'req-c5f1e7eb-460e-4792-8e15-05c7e517629c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:51.273981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] The token cache is being invalidate because project 4c7ea08516cf4ca1801d1f1f82804b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:51.274373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:51.275298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-c5f1e7eb-460e-4792-8e15-05c7e517629c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:51.280438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1070/2077] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:51 2022] DELETE /identity/v3/projects/4c7ea08516cf4ca1801d1f1f82804b0a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:51.286466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.287060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.288070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.334017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:51.335518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:51.440129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1008/2078] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:51.448294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.449085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.449527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.480590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:51.483588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:51.495316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1071/2079] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:51.502101 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.513649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58310d4b-f46d-4036-98ca-633f00a45049 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.521453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.522172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.522549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.522829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.523606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:51.523947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.526345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.554481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1009/2080] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:51.557124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.574355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a4eb-f26f-4b05-ad7b-96b155c21e6c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.580870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.582066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.582495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.583015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.583873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.586383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.624836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1072/2081] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:51 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:51.630683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.644077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14308a0d-65bf-41bc-87d2-5654be8bfe2e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.692448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:51.722620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.723263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.723540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.723852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.724535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.727008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.756162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1010/2082] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:51 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:40:51.760702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.773585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b60149-40bc-4fb0-9de2-97b6a7c4e0aa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.779807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.780558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.780991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.781281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.782179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.785449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.799945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1073/2083] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:51.804673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.816985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e97f27-2a0c-4ab7-b7e2-3f9c14b64950 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.822935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a50aa5-af63-44c8-89cd-d5b3dfa8a5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.823580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a50aa5-af63-44c8-89cd-d5b3dfa8a5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.823894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a50aa5-af63-44c8-89cd-d5b3dfa8a5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.824163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a50aa5-af63-44c8-89cd-d5b3dfa8a5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.824972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a50aa5-af63-44c8-89cd-d5b3dfa8a5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.826797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a50aa5-af63-44c8-89cd-d5b3dfa8a5c1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.843354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1011/2084] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:51.844946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:51.856640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6df6-1331-4a34-9cfb-cf04f5c16864 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:51.861860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:51.862569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.862940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.863811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] PATH_INFO: `/v3/projects/38a64972abf34a148754a8e8ed9960d8/users/f51dfeb220b74a7788aecacb6a24162c/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.883163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a64972abf34a148754a8e8ed9960d8, user_id=f51dfeb220b74a7788aecacb6a24162c, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:51.886530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:51.914273 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1074/2085] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:51 2022] PUT /identity/v3/projects/38a64972abf34a148754a8e8ed9960d8/users/f51dfeb220b74a7788aecacb6a24162c/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:51.934319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:51.934602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:51.934883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:51.973330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] MFA Rules not processed for user `f51dfeb220b74a7788aecacb6a24162c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:51.974931 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:52.071026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1012/2086] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:52.075155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.172452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a06ee4-501f-4d7b-baa0-a549ed351c71 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.177531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.178318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.178599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.178875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.180020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.182408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.223827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1075/2087] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:52 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:52.227097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.241815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728758f-f90f-4bcd-b709-c07b97aeee5b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.334287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.335055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.335416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.335919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.336839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.339012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.366512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1013/2088] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:52 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:52.370398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.381696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2e882-e33c-4405-8ff9-5e07254956b9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.387006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.387829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.388171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.388509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.389281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.390963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.407865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1076/2089] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:52.410359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.422241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca5e5-e641-47f2-9e2a-005075484878 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.428433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.429199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.429463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.429743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] PATH_INFO: `/v3/projects/43865ce4aec54bfcaa10db576663721c/users/9e8a2775b7ce4f1598356d7072db4822/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.480837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=43865ce4aec54bfcaa10db576663721c, user_id=9e8a2775b7ce4f1598356d7072db4822, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.484013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.505523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1014/2090] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:52 2022] PUT /identity/v3/projects/43865ce4aec54bfcaa10db576663721c/users/9e8a2775b7ce4f1598356d7072db4822/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:52.506133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.521452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b3fc5-007f-4b6e-909a-af1ccfa8d72a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.586793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:52.597876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039c1a2-4208-44d6-a533-c338e13cf7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.598621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039c1a2-4208-44d6-a533-c338e13cf7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.598873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039c1a2-4208-44d6-a533-c338e13cf7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.599148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039c1a2-4208-44d6-a533-c338e13cf7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.600038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039c1a2-4208-44d6-a533-c338e13cf7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.601997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039c1a2-4208-44d6-a533-c338e13cf7d7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.613373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1077/2091] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:52.617611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.630877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565596-3192-4127-81a1-bb2c3db667ca admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.637138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.637940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.638444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.638728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] PATH_INFO: `/v3/projects/43865ce4aec54bfcaa10db576663721c/users/9e8a2775b7ce4f1598356d7072db4822/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.642268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43865ce4aec54bfcaa10db576663721c, user_id=9e8a2775b7ce4f1598356d7072db4822, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.645517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.655419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:52.677891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1015/2092] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:52 2022] PUT /identity/v3/projects/43865ce4aec54bfcaa10db576663721c/users/9e8a2775b7ce4f1598356d7072db4822/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:52.678997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.679461 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.679735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.715441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] MFA Rules not processed for user `9e8a2775b7ce4f1598356d7072db4822`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:52.716758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:52.826008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1078/2093] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:52.840434 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.842949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:52.865512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:52.867063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:52.919060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf643-96f1-4840-930f-6b62fa3112f4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.923062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80cd3-986c-4f63-b747-d436c2c8ee22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.924041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80cd3-986c-4f63-b747-d436c2c8ee22 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.924450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80cd3-986c-4f63-b747-d436c2c8ee22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.924799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80cd3-986c-4f63-b747-d436c2c8ee22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.934753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba80cd3-986c-4f63-b747-d436c2c8ee22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.936363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba80cd3-986c-4f63-b747-d436c2c8ee22 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:52.964284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c2736-8332-4e58-a851-22d2d74cc49c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:52.973220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:52.974216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:52.974359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:52.974595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] PATH_INFO: `/v3/users/122867e95b9a4e60861354f750ad01d7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:52.976038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1016/2094] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:52.976873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] RBAC: Authorizing `identity:delete_user(user_id=122867e95b9a4e60861354f750ad01d7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:52.979040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:53.080639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122867e95b9a4e60861354f750ad01d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.081630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122867e95b9a4e60861354f750ad01d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.095285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122867e95b9a4e60861354f750ad01d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.115968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122867e95b9a4e60861354f750ad01d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.122721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:53.126338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122867e95b9a4e60861354f750ad01d7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.136726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1079/2095] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:52 2022] DELETE /identity/v3/users/122867e95b9a4e60861354f750ad01d7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:53.141233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:53.141525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:53.141810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:53.161896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:53.163209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.276346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1017/2096] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:53.284967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:53.287116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.388923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6edfb-58ad-4614-b62e-b75dbba3729c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:53.393842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:53.394487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:53.394767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:53.395040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:53.406036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:53.407565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:53.419405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1080/2097] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:53.701568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:53.702994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.716135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e655696-e8c9-4052-be10-39dd8631b50a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:53.722057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:53.722960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:53.723243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:53.723516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] PATH_INFO: `/v3/users/f51dfeb220b74a7788aecacb6a24162c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:53.724917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51dfeb220b74a7788aecacb6a24162c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:53.727392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:53.741950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:53.743735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.759315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec39fa-91c8-4838-9c77-65cbcf139a2f service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:53.764194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:53.764940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:53.765226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:53.765533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] PATH_INFO: `/v3/projects/b39aa2618f634a3d8e9a98703f8da477` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:53.767036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b39aa2618f634a3d8e9a98703f8da477)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:53.770432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:53.794326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51dfeb220b74a7788aecacb6a24162c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.794948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51dfeb220b74a7788aecacb6a24162c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.808984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51dfeb220b74a7788aecacb6a24162c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.816461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51dfeb220b74a7788aecacb6a24162c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.817155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-94dc4196-7b87-45d6-8333-9d27c38bccf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51dfeb220b74a7788aecacb6a24162c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.824117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b39aa2618f634a3d8e9a98703f8da477', 'request_id': 'req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.831163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b39aa2618f634a3d8e9a98703f8da477', 'request_id': 'req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.831881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1018/2098] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:53 2022] DELETE /identity/v3/users/f51dfeb220b74a7788aecacb6a24162c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:53.836957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:53.837378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:53.837792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:53.839357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b39aa2618f634a3d8e9a98703f8da477', 'request_id': 'req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.852462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] The token cache is being invalidate because project b39aa2618f634a3d8e9a98703f8da477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:53.852651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.853300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.854832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1081/2099] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:53 2022] DELETE /identity/v3/projects/b39aa2618f634a3d8e9a98703f8da477 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:53.858578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:53.859064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:53.859818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.860145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.869426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:53.926400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1019/2100] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:53.928833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d76cb-3f6d-4efd-9254-dc19eb797ff9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:53.933658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:53.934398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:53.934674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:53.934943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] PATH_INFO: `/v3/users/8ea2b917802246b58423e1d051af14db` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:53.935286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:53.936253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea2b917802246b58423e1d051af14db)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:53.936680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:53.938177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:53.962574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:53.992229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea2b917802246b58423e1d051af14db'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:53.993135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea2b917802246b58423e1d051af14db'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.001775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1329-9c97-4938-bb35-5b1a6a4054fd None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:54.003594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea2b917802246b58423e1d051af14db'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.005884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:54.006517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:54.006795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:54.007065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:54.016974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea2b917802246b58423e1d051af14db'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.017219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:54.019597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:54.025281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d876cf3-2ec0-4b51-b8ea-33d65acbd681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea2b917802246b58423e1d051af14db'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.030117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1020/2101] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:40:54.036103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1082/2102] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:53 2022] DELETE /identity/v3/users/8ea2b917802246b58423e1d051af14db => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:54.489933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:54.491416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:54.541675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9060d-9440-446d-844c-1f3548f9ed67 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:54.612057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:54.612349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:54.612603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:54.654428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:54.662137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:54.670758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:54.671384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:54.672538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:54.672875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] PATH_INFO: `/v3/projects/38a64972abf34a148754a8e8ed9960d8` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:54.675265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a64972abf34a148754a8e8ed9960d8)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:54.680808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:54.756708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a64972abf34a148754a8e8ed9960d8', 'request_id': 'req-07be467f-c8a8-4315-8618-ab8216f828bd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.763565 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a64972abf34a148754a8e8ed9960d8', 'request_id': 'req-07be467f-c8a8-4315-8618-ab8216f828bd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.765686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:54.775310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a64972abf34a148754a8e8ed9960d8', 'request_id': 'req-07be467f-c8a8-4315-8618-ab8216f828bd'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.785848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1083/2103] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:40:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:40:54.786967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:54.788493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:54.801802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] The token cache is being invalidate because project 38a64972abf34a148754a8e8ed9960d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:54.802135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.802749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:54.805608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1021/2104] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:54 2022] DELETE /identity/v3/projects/38a64972abf34a148754a8e8ed9960d8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:54.806463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:54.808002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:54.811473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c2357-a35d-4f3e-9d5e-d887b43868ea None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:54.921103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be467f-c8a8-4315-8618-ab8216f828bd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:54.932442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:54.933117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:54.936185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:54.936852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:54.940059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:54.944389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:54.946120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:54.946684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:54.947082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] PATH_INFO: `/v3/projects/ce46d9fa1a304210b1b795be0356a7dd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:54.949082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce46d9fa1a304210b1b795be0356a7dd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:54.951263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:55.030220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:55.036623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce46d9fa1a304210b1b795be0356a7dd', 'request_id': 'req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.057745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce46d9fa1a304210b1b795be0356a7dd', 'request_id': 'req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.069041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce46d9fa1a304210b1b795be0356a7dd', 'request_id': 'req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.089160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] The token cache is being invalidate because project ce46d9fa1a304210b1b795be0356a7dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:55.089522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.091003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.092993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1084/2105] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:54 2022] DELETE /identity/v3/projects/ce46d9fa1a304210b1b795be0356a7dd => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:55.097286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:55.098193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:55.101044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:55.101547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0175c1b-4701-4f5f-adcf-82bf4e2629be service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:55.126050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1022/2106] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:40:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:40:55.128362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ff24-e604-45d0-ae40-43db77bf0fa1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:55.128362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ff24-e604-45d0-ae40-43db77bf0fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:55.128543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ff24-e604-45d0-ae40-43db77bf0fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:55.163721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5691ff24-e604-45d0-ae40-43db77bf0fa1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:55.165152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5691ff24-e604-45d0-ae40-43db77bf0fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:55.213748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8f5cf-f01f-464c-87ce-ebdf6778c8c7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:55.238515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:55.240107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:55.240311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:55.240592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] PATH_INFO: `/v3/users/9e8a2775b7ce4f1598356d7072db4822` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:55.245056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8a2775b7ce4f1598356d7072db4822)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:55.249234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:55.307874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1023/2107] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:55.315909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:55.317069 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:55.317543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:55.372230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8a2775b7ce4f1598356d7072db4822'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.372737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:55.373341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8a2775b7ce4f1598356d7072db4822'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.375852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:55.394909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8a2775b7ce4f1598356d7072db4822'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.404097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1024/2108] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:55.408972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:55.433825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74dbc-cc60-4086-acb6-3a677ab72858 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:55.452304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8a2775b7ce4f1598356d7072db4822'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.456261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:55.457963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:55.458542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:55.459090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:55.460163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:55.460845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:55.462797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:55.476781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8a2775b7ce4f1598356d7072db4822'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:55.483494 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1025/2109] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:55.487027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:55.494013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:55.499730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1085/2110] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:40:55 2022] DELETE /identity/v3/users/9e8a2775b7ce4f1598356d7072db4822 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:55.527669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aabf46-d870-4794-a822-d68108223557 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:55.537557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:55.716304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:55.717066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:55.717527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:55.717799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:55.718492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:55.724115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:55.842912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1026/2111] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:55 2022] POST /identity/v3/projects => generated 363 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:55.844742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:55.863837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d2550-650a-4406-bd1a-e80501784a21 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:55.998328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:56.023666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.023957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.024245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.024610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.025722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.035567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.092352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1086/2112] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:55 2022] POST /identity/v3/users => generated 282 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:56.094074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.110983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dcb50-e808-439d-b03f-551aa8ced6ff admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.119894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.120648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.120930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.121275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.122082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.123964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.141942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1027/2113] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:56.143917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.155564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867ef54-a62f-49ed-b4ac-761a81705eec admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.161707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc2359-f97b-4b64-8f58-98ad0f4777ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.162422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc2359-f97b-4b64-8f58-98ad0f4777ff admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.162727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc2359-f97b-4b64-8f58-98ad0f4777ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.163125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc2359-f97b-4b64-8f58-98ad0f4777ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.163936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc2359-f97b-4b64-8f58-98ad0f4777ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.166225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc2359-f97b-4b64-8f58-98ad0f4777ff admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.182519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1087/2114] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:56.184499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.198216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a598-c1f6-4c1d-b860-bcfcc19e6c08 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.207903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.209715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.210346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.210898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] PATH_INFO: `/v3/projects/85610d78403f4101b3589cac08212d4d/users/e9943daa89804d6fb771755e74f10cb2/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.248661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85610d78403f4101b3589cac08212d4d, user_id=e9943daa89804d6fb771755e74f10cb2, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.251529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.280283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1028/2115] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:56 2022] PUT /identity/v3/projects/85610d78403f4101b3589cac08212d4d/users/e9943daa89804d6fb771755e74f10cb2/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:56.293778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.294248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.294361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.327211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] MFA Rules not processed for user `e9943daa89804d6fb771755e74f10cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:56.328796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:56.462035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1088/2116] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:56 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:56.466891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.570658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.572321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:56.626014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b76f7-4f36-4f51-91e3-dc2f2ec1a65e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.634356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.635974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.638301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.638694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.643973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.650342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.731887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1029/2117] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:56 2022] POST /identity/v3/projects => generated 363 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:56.733084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.778100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9e751-f06e-455e-a68d-2dd3d2908e3a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.804366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:56.846319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ecd5e-77d4-4753-ac56-58c9a97228c6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.869800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.871915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.872488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.874550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.879893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.888157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.893716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.894463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.894808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.895241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] PATH_INFO: `/v3/projects/43865ce4aec54bfcaa10db576663721c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.896622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43865ce4aec54bfcaa10db576663721c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.898958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:56.914856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:56.930057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1030/2118] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:56 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:56.935707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:56.955591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806800e-133c-4816-a6c7-ba81e71a9aa2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:56.974621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:56.975611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:56.975920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:56.976186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:56.976897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:56.978540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:57.000034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43865ce4aec54bfcaa10db576663721c', 'request_id': 'req-a6981f3b-e37d-457d-8519-6766770fb782'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:57.004687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1031/2119] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:57.008835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.011873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43865ce4aec54bfcaa10db576663721c', 'request_id': 'req-a6981f3b-e37d-457d-8519-6766770fb782'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:57.025236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43865ce4aec54bfcaa10db576663721c', 'request_id': 'req-a6981f3b-e37d-457d-8519-6766770fb782'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:57.039904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b103b-7651-4f64-84b3-0f6a8681bd6f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:57.062561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] The token cache is being invalidate because project 43865ce4aec54bfcaa10db576663721c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:40:57.064008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:57.065048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a6981f3b-e37d-457d-8519-6766770fb782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:40:57.067216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1089/2120] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:40:56 2022] DELETE /identity/v3/projects/43865ce4aec54bfcaa10db576663721c => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:40:57.072839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de458b3-5a31-4487-a911-20d3ea289195 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.073511 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de458b3-5a31-4487-a911-20d3ea289195 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.074193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de458b3-5a31-4487-a911-20d3ea289195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.115211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3de458b3-5a31-4487-a911-20d3ea289195 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:57.116583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3de458b3-5a31-4487-a911-20d3ea289195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:57.176634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:57.177928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.178505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.180493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] PATH_INFO: `/v3/projects/8071d7f7238643ac953434db3c43e680/users/082eb0a2b2534df78ed1232c3ea85858/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.217751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8071d7f7238643ac953434db3c43e680, user_id=082eb0a2b2534df78ed1232c3ea85858, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:57.225727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:57.232939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1090/2121] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:57.243087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.244350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.246917 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.261972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1032/2122] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:57 2022] PUT /identity/v3/projects/8071d7f7238643ac953434db3c43e680/users/082eb0a2b2534df78ed1232c3ea85858/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:57.264485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.269863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:57.299071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:57.304886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:57.501178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65647-c5f8-4e2e-99b3-ffc820ae9527 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:57.515884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:57.515884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.515884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.515884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.516366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:57.518286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:57.523301 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1091/2123] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:57.525738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.541742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffee133-b7d9-479e-978b-270a67951c1b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:57.545705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1033/2124] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:40:57.550418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.554247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:57.557019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.557019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.557019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.557019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:57.557019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:57.558265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:57.579154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4522c84-4431-4ed6-89f4-30063bc34ed1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:57.590764 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1092/2125] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:57.594085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.605242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84db0af-f0e7-4a1c-a6c1-465e8000e3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:57.609090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84db0af-f0e7-4a1c-a6c1-465e8000e3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.609402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84db0af-f0e7-4a1c-a6c1-465e8000e3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.609678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84db0af-f0e7-4a1c-a6c1-465e8000e3f2 admin admin] PATH_INFO: `/v3/projects/8071d7f7238643ac953434db3c43e680/users/082eb0a2b2534df78ed1232c3ea85858/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.613364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84db0af-f0e7-4a1c-a6c1-465e8000e3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8071d7f7238643ac953434db3c43e680, user_id=082eb0a2b2534df78ed1232c3ea85858, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:57.616966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84db0af-f0e7-4a1c-a6c1-465e8000e3f2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:57.617459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2b1d-5359-4202-978a-bb72354ad5a7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:57.627962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:57.629503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.630070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.630751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.632098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:57.642701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:57.644326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1034/2126] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:57 2022] PUT /identity/v3/projects/8071d7f7238643ac953434db3c43e680/users/082eb0a2b2534df78ed1232c3ea85858/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:57.646226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.646748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.647201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:57.708624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] MFA Rules not processed for user `082eb0a2b2534df78ed1232c3ea85858`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:57.711121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:57.741405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1093/2127] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:40:57 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:57.748425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.760055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:57.776234 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:57.943481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1035/2128] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:57.950926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:57.952397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:57.981287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a182f8-0552-41c7-9e89-e5e40c1d1f14 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:57.997687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:57.998675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:57.999170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:57.999628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.000567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.003570 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.065837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5b313-e33c-4d61-a430-4efc6756ac60 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:58.073784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1094/2129] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:57 2022] POST /identity/v3/users => generated 286 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:58.085071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:58.110427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f283947-73f8-4071-8e08-a7917389dbf4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:58.115788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:58.116582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.116710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.116967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.117777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.119326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.139349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1095/2130] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:58.143569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:58.146877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a231d-79ed-48ad-a1cc-9045a3222c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:58.147521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a231d-79ed-48ad-a1cc-9045a3222c88 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.147832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a231d-79ed-48ad-a1cc-9045a3222c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.148109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a231d-79ed-48ad-a1cc-9045a3222c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.157624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb56bb-d2bb-4a1a-837d-1cbe104fab59 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:58.161634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:58.162751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.163997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.164460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.165627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.167680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.188457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1096/2131] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:58.195047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:58.217351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30c5ab-217a-4c65-8162-0627b586372b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:58.234150 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:58.234833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.235113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.235444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] PATH_INFO: `/v3/projects/02c82e1afd544a8c8b64d8ddb13028bc/users/21f622f2e5d8419d9f56a17a48dd89bb/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.277882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c82e1afd544a8c8b64d8ddb13028bc, user_id=21f622f2e5d8419d9f56a17a48dd89bb, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.281498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a231d-79ed-48ad-a1cc-9045a3222c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.285182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.289415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a231d-79ed-48ad-a1cc-9045a3222c88 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.306328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1036/2132] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:58.308983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1097/2133] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:40:58 2022] PUT /identity/v3/projects/02c82e1afd544a8c8b64d8ddb13028bc/users/21f622f2e5d8419d9f56a17a48dd89bb/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:58.314672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.315134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.315294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.346732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:58.348017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:58.364788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:58.421752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1037/2134] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:40:58.433109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:58.434934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:58.438973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:58.526521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477efe1-a49f-43c0-9854-1d24081b8ad0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:58.527261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db599fe-93d4-4b6b-9cbd-21bfab72cd63 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:58.530589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:58.531440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.531900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.532058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373da2c-90eb-4646-bf39-b0a718c3fcef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:58.532194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.532659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373da2c-90eb-4646-bf39-b0a718c3fcef service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:58.533012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373da2c-90eb-4646-bf39-b0a718c3fcef service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:58.533959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373da2c-90eb-4646-bf39-b0a718c3fcef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:58.534166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:58.556939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373da2c-90eb-4646-bf39-b0a718c3fcef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.560682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373da2c-90eb-4646-bf39-b0a718c3fcef service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.579481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1098/2135] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:40:58.608718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:58.610217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:58.624815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1038/2136] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:40:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:41:00.069212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.070378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.071488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.138242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] MFA Rules not processed for user `21f622f2e5d8419d9f56a17a48dd89bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:00.145733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:00.219575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:00.282083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1099/2137] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:00.287750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:00.303738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b01dc-57bc-493b-a68b-31dabf462146 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:00.309161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:00.309802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.310084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.310351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.311033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:00.313354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:00.364280 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1039/2138] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:00 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:00.368369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:00.384086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f0be0-ab26-4aac-8f1b-9cbc492401ce None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:00.477398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:00.478036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.478310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.478575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.479248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:00.481385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:00.516445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1100/2139] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:00 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:41:00.530675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:00.544798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35fbb3-43a9-46a1-83e2-bf8711c83d20 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:00.552813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:00.553691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.554332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.554843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.555829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:00.559467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:00.574105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1040/2140] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:00.578151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:00.608153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c57c8b-3bfa-4454-82e1-c56905f79ec7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:00.615036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:00.615819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.616233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.616353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] PATH_INFO: `/v3/projects/c2b463dd05fb4b5e88d7ca74ea22ab97/users/b1285f18115144d591be2db3b630bb08/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.679364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b463dd05fb4b5e88d7ca74ea22ab97, user_id=b1285f18115144d591be2db3b630bb08, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:00.690487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:00.717209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1101/2141] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:00 2022] PUT /identity/v3/projects/c2b463dd05fb4b5e88d7ca74ea22ab97/users/b1285f18115144d591be2db3b630bb08/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:00.722051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:00.733308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef96da-fe05-4d4f-b87a-cca659fdcdcd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:00.827330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:00.856659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086ada6-f33b-42b8-8320-1b8d94954472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:00.857667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086ada6-f33b-42b8-8320-1b8d94954472 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.857926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086ada6-f33b-42b8-8320-1b8d94954472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.858196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086ada6-f33b-42b8-8320-1b8d94954472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.858926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086ada6-f33b-42b8-8320-1b8d94954472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:00.860571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086ada6-f33b-42b8-8320-1b8d94954472 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:00.872739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1041/2142] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:00.876964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:00.890599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658fb58-8bb6-4d65-86fa-b3a9c5281a96 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:00.913200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:00.913942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:00.914226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:00.914512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] PATH_INFO: `/v3/projects/c2b463dd05fb4b5e88d7ca74ea22ab97/users/b1285f18115144d591be2db3b630bb08/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:00.916645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b463dd05fb4b5e88d7ca74ea22ab97, user_id=b1285f18115144d591be2db3b630bb08, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:00.919378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:00.941304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1102/2143] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:00 2022] PUT /identity/v3/projects/c2b463dd05fb4b5e88d7ca74ea22ab97/users/b1285f18115144d591be2db3b630bb08/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:41:01.829302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:01.830003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:01.830124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:01.879430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:01.919979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] MFA Rules not processed for user `b1285f18115144d591be2db3b630bb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:01.919979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:02.021203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d3d15-2d76-47f7-b527-1acb4f2c087b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:02.032290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:02.033016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:02.033295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:02.033569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] PATH_INFO: `/v3/users/e9943daa89804d6fb771755e74f10cb2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:02.035737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9943daa89804d6fb771755e74f10cb2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:02.038739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:02.079973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1042/2144] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:02.093065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:02.159502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:02.210684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9943daa89804d6fb771755e74f10cb2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:02.213199 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9943daa89804d6fb771755e74f10cb2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:02.231277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9943daa89804d6fb771755e74f10cb2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:02.248090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9943daa89804d6fb771755e74f10cb2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:02.263877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f91ae-609a-4e5d-8664-cbc93daa6b22 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:02.270580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fa5defcb-5279-4a5e-afa5-b0aa545fdce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9943daa89804d6fb771755e74f10cb2'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:02.274334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:02.276943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:02.277593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:02.281706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1103/2145] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:01 2022] DELETE /identity/v3/users/e9943daa89804d6fb771755e74f10cb2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:41:02.281896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:02.284970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:02.285846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:02.286155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:02.286431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:02.330170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:02.331206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:02.458612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1104/2146] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:41:02.464954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:02.467522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:02.468296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:02.469850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:02.492968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1043/2147] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:02.630826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcde85-f935-422c-8dc6-fa79aa68b195 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:02.640038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:02.640830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:02.641208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:02.641578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:02.667608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:02.669233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:02.699849 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1105/2148] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:41:03.285879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:03.287578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:03.312083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b0139-5444-4ad9-8b49-d8eca3d21604 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:03.328975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:03.332108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:03.332539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:03.332941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] PATH_INFO: `/v3/projects/85610d78403f4101b3589cac08212d4d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:03.334456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85610d78403f4101b3589cac08212d4d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:03.337490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:03.356062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:03.407587 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85610d78403f4101b3589cac08212d4d', 'request_id': 'req-9c710514-0636-450c-be69-3d85683af2d8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.420914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85610d78403f4101b3589cac08212d4d', 'request_id': 'req-9c710514-0636-450c-be69-3d85683af2d8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.440699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85610d78403f4101b3589cac08212d4d', 'request_id': 'req-9c710514-0636-450c-be69-3d85683af2d8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.468065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] The token cache is being invalidate because project 85610d78403f4101b3589cac08212d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:03.468408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.469081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.471257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1044/2149] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:03 2022] DELETE /identity/v3/projects/85610d78403f4101b3589cac08212d4d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:03.475626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:03.506982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ae213-252c-452e-b864-b9f65b71c2b1 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:03.711593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:03.738768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:03.740079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:03.740673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:03.741285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] PATH_INFO: `/v3/users/082eb0a2b2534df78ed1232c3ea85858` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:03.747484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=082eb0a2b2534df78ed1232c3ea85858)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:03.751033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:03.893438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082eb0a2b2534df78ed1232c3ea85858'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.899802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082eb0a2b2534df78ed1232c3ea85858'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.925571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082eb0a2b2534df78ed1232c3ea85858'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.945887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082eb0a2b2534df78ed1232c3ea85858'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.968740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082eb0a2b2534df78ed1232c3ea85858'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:03.981495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1106/2150] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:03 2022] DELETE /identity/v3/users/082eb0a2b2534df78ed1232c3ea85858 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:04.387151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:04.390032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:04.448290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:04.450525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:04.494198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710514-0636-450c-be69-3d85683af2d8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:04.553027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:04.553668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:04.553939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:04.554363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:04.556309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:04.572613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053eb1-0e24-4556-89ff-bb16bbcca6dc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:04.577604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:04.578256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:04.578536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:04.578805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] PATH_INFO: `/v3/projects/8071d7f7238643ac953434db3c43e680` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:04.580518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8071d7f7238643ac953434db3c43e680)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:04.582430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:04.624029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:04.625402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e8aab-9019-4d10-b65e-998da0faca38 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:04.635288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8071d7f7238643ac953434db3c43e680', 'request_id': 'req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:04.636493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1045/2151] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:04.644034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8071d7f7238643ac953434db3c43e680', 'request_id': 'req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:04.653582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8071d7f7238643ac953434db3c43e680', 'request_id': 'req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:04.666988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] The token cache is being invalidate because project 8071d7f7238643ac953434db3c43e680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:04.667275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:04.673260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3eafb68b-1fa5-45b5-8382-70b5c84c2a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:04.673260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1107/2152] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:04 2022] DELETE /identity/v3/projects/8071d7f7238643ac953434db3c43e680 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:04.678378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:04.678667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:04.678942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:04.693985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:04.701648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:04.703483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:04.766412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1046/2153] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:04.774662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:04.774988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:04.775392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:04.786545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:04.797490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:04.800563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:04.810541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1108/2154] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:04.814198 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:04.826584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3b949-7965-400e-99eb-1772e0580251 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:04.840104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:04.840104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:04.840104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:04.840104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:04.840417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:41:04.840417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:04.840417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:04.854610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1047/2155] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:41:04.859634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:04.870038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d38e-8e03-4fc9-822d-627ab23ce8a3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:04.875295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:04.875904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:04.876309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:04.876937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:04.878266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:04.880228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:04.914520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1109/2156] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:04 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:41:04.918619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:04.933216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513dd61-7965-433a-8619-a56e93b1dbfc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.015885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.017065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.017435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.017845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.018676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.020762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.071125 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.071352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1048/2157] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:04 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:05.086856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4810c9-1e41-4284-941b-7165ccabbdf2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.092571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.093738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.094432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.094770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.096015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.098674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.118866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1110/2158] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:41:05.119291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.134546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfc0ea-623e-4857-84c5-b824f3e6310c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.148667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a05b0d-23a4-4e28-a20d-05694e8dbd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.149327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a05b0d-23a4-4e28-a20d-05694e8dbd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.149647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a05b0d-23a4-4e28-a20d-05694e8dbd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.149916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a05b0d-23a4-4e28-a20d-05694e8dbd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.150722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a05b0d-23a4-4e28-a20d-05694e8dbd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.155219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a05b0d-23a4-4e28-a20d-05694e8dbd47 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.182916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.187949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1049/2159] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:05.199331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad2151-20ae-4586-84c4-072646072efe admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.232669 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.233936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.235069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.236124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] PATH_INFO: `/v3/projects/7804cbe463ba4213b6b2a5c12cd14590/users/be5398e336a540249f4173cc0177db7f/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.278041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7804cbe463ba4213b6b2a5c12cd14590, user_id=be5398e336a540249f4173cc0177db7f, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.281522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.302081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1111/2160] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:05 2022] PUT /identity/v3/projects/7804cbe463ba4213b6b2a5c12cd14590/users/be5398e336a540249f4173cc0177db7f/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:41:05.313940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.314386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.314782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.380497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] MFA Rules not processed for user `be5398e336a540249f4173cc0177db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:05.382975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:05.516277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1050/2161] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:05.520970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.595743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d76f8-39f7-4f2d-947a-8a4944b35801 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.613222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.613222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.613222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.613222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.613581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.613581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.635470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1112/2162] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:05 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:05.639959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.646444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:05.653367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a883a3-31c3-42e5-beb2-3d0202f38111 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.714442 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.715212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.715693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.716237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.717512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.719994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.742473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1051/2163] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:05 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:41:05.746604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.757157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc51c98-b503-46b9-9c07-922f02fca73f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.762605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.763262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.763541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.763846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.764552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.766197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.777470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1113/2164] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:05.781829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.793253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d2e7a-afcf-4e7d-851b-7a4bbc52c201 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.798533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.799321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.799598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.799907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] PATH_INFO: `/v3/projects/f398c6b160b44386b3835454e3e8e1db/users/3a6458d98f0a4ebb828adfb8cfdc7562/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.816451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f398c6b160b44386b3835454e3e8e1db, user_id=3a6458d98f0a4ebb828adfb8cfdc7562, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.819661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.835372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1052/2165] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:05 2022] PUT /identity/v3/projects/f398c6b160b44386b3835454e3e8e1db/users/3a6458d98f0a4ebb828adfb8cfdc7562/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:05.839485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.850079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490933e-b4db-465f-b883-90badd028dc4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.910007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161748e-0e11-49eb-800e-945767cc7921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.910544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161748e-0e11-49eb-800e-945767cc7921 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.910818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161748e-0e11-49eb-800e-945767cc7921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.911091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161748e-0e11-49eb-800e-945767cc7921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.911853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161748e-0e11-49eb-800e-945767cc7921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.913778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161748e-0e11-49eb-800e-945767cc7921 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.924541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1114/2166] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:05.928793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:05.939735 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0806-d50a-451e-b39d-4ccfb0421c89 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:05.945020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:05.945712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.946007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.946275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] PATH_INFO: `/v3/projects/f398c6b160b44386b3835454e3e8e1db/users/3a6458d98f0a4ebb828adfb8cfdc7562/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.948325 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f398c6b160b44386b3835454e3e8e1db, user_id=3a6458d98f0a4ebb828adfb8cfdc7562, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:05.951373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:05.962835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1053/2167] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:05 2022] PUT /identity/v3/projects/f398c6b160b44386b3835454e3e8e1db/users/3a6458d98f0a4ebb828adfb8cfdc7562/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:05.973311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:05.973597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:05.973861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:05.999251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:06.011613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] MFA Rules not processed for user `3a6458d98f0a4ebb828adfb8cfdc7562`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:06.012908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:06.105720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1115/2168] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:06.115084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:06.117632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:06.194910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ad1d1-8d86-4c03-8f74-2ee601a6fd01 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:06.198981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:06.199646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:06.199939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:06.200224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:06.213072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:06.215136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:06.231020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1054/2169] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:06.345656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:06.347271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:06.364618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a9c67-0023-4498-9450-d3d8cb485e24 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:06.370012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81148cac-b126-473d-9e6e-d6676684afe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:06.370800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81148cac-b126-473d-9e6e-d6676684afe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:06.371190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81148cac-b126-473d-9e6e-d6676684afe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:06.371590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81148cac-b126-473d-9e6e-d6676684afe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:06.488412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81148cac-b126-473d-9e6e-d6676684afe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:06.490207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81148cac-b126-473d-9e6e-d6676684afe9 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:06.512550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1116/2170] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:06.955856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:07.024482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:07.033896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587127f-c46e-4134-b683-a10e508d5a83 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:07.042216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:07.043045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:07.043421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:07.043980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] PATH_INFO: `/v3/users/be5398e336a540249f4173cc0177db7f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:07.046131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5398e336a540249f4173cc0177db7f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:07.050045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:07.108984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5398e336a540249f4173cc0177db7f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.109584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5398e336a540249f4173cc0177db7f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.121872 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5398e336a540249f4173cc0177db7f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.129190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5398e336a540249f4173cc0177db7f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.129938 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5398e336a540249f4173cc0177db7f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.147036 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1055/2171] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:06 2022] DELETE /identity/v3/users/be5398e336a540249f4173cc0177db7f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:07.153168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:07.153884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:07.154057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:07.176637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:07.177875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:07.268446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1117/2172] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:07.282116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:07.284079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:07.353084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89577751-f964-45ec-89ef-1e6086f47cc0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:07.357098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:07.357731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:07.357997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:07.358286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:07.368515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:07.369982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:07.382516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1056/2173] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:07.830046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:07.831908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:07.846590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754951d-7429-483f-bd24-bb3869fddb4c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:07.852780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:07.853978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:07.856632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:07.856632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] PATH_INFO: `/v3/projects/7804cbe463ba4213b6b2a5c12cd14590` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:07.856632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7804cbe463ba4213b6b2a5c12cd14590)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:07.865633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:07.914740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:07.930022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7804cbe463ba4213b6b2a5c12cd14590', 'request_id': 'req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.936336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7804cbe463ba4213b6b2a5c12cd14590', 'request_id': 'req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.944889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7804cbe463ba4213b6b2a5c12cd14590', 'request_id': 'req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.956539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] The token cache is being invalidate because project 7804cbe463ba4213b6b2a5c12cd14590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:07.956818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.957468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:07.958992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1118/2174] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:07 2022] DELETE /identity/v3/projects/7804cbe463ba4213b6b2a5c12cd14590 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:07.963291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:07.974123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706c12-8589-4ef8-9695-2d6de21cb2d0 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:08.040759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:08.042522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:08.043192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:08.043660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] PATH_INFO: `/v3/users/3a6458d98f0a4ebb828adfb8cfdc7562` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:08.045187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6458d98f0a4ebb828adfb8cfdc7562)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:08.048114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:08.078255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:08.130969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6458d98f0a4ebb828adfb8cfdc7562'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:08.132612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6458d98f0a4ebb828adfb8cfdc7562'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:08.170980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6458d98f0a4ebb828adfb8cfdc7562'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:08.192649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6458d98f0a4ebb828adfb8cfdc7562'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:08.196262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-579becf2-d55d-4cef-8f9b-a024e86deb5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6458d98f0a4ebb828adfb8cfdc7562'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:08.231628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1057/2175] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:07 2022] DELETE /identity/v3/users/3a6458d98f0a4ebb828adfb8cfdc7562 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:08.788718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:08.790401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:08.810747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df8c6-b10c-4d92-a17c-1bec860c9dd9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:08.887027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:08.887701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:08.888058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:08.888329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] PATH_INFO: `/v3/projects/f398c6b160b44386b3835454e3e8e1db` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:08.889513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f398c6b160b44386b3835454e3e8e1db)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:08.891785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:08.977850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f398c6b160b44386b3835454e3e8e1db', 'request_id': 'req-fc150579-293b-43e5-b0a5-875e389daf97'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:08.986731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f398c6b160b44386b3835454e3e8e1db', 'request_id': 'req-fc150579-293b-43e5-b0a5-875e389daf97'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:09.004163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f398c6b160b44386b3835454e3e8e1db', 'request_id': 'req-fc150579-293b-43e5-b0a5-875e389daf97'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:09.023212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] The token cache is being invalidate because project f398c6b160b44386b3835454e3e8e1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:09.023992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:09.024851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-fc150579-293b-43e5-b0a5-875e389daf97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:09.026505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1119/2176] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:08 2022] DELETE /identity/v3/projects/f398c6b160b44386b3835454e3e8e1db => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:09.032347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.033311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.033591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.056776 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:09.058267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:09.135829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1058/2177] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:09.145674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.146042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.146385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.167904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:09.169133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:09.177453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1120/2178] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:09.182024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.191997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422e7bb-f8f3-4881-bbb1-07ca45d96fff None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.197232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.197873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.198388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.198733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.199571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:41:09.199982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.201779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.218758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1059/2179] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:41:09.220727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.231083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7a03-1c4c-4cec-b717-f174ff2145aa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.236277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.236968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.237275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.237534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.238244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.240246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.271008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1121/2180] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:09 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:41:09.275548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.287667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8bed3-882b-4ea3-8440-21cdaa9b2ee3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.354910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.355534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.355845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.356248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.356968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.359050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.401982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1060/2181] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:09 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:41:09.408699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.456823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fe6b-3e0e-43c1-adb4-4fe53fa66b3c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.464353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.465342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.465762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.466250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.467397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.469297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.491801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1122/2182] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:09.495892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.510825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e2516-1899-4f4a-b719-4ee7f9ae15cb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.519593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728c35c-7e44-4054-9625-6d347599ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.520569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728c35c-7e44-4054-9625-6d347599ec1b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.520686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728c35c-7e44-4054-9625-6d347599ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.520902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728c35c-7e44-4054-9625-6d347599ec1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.521649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728c35c-7e44-4054-9625-6d347599ec1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.523853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728c35c-7e44-4054-9625-6d347599ec1b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.540903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1061/2183] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:09.545112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.555818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769016ae-fd4c-4c50-bf9f-3787ede129b3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.560820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.562338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.562338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.562338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] PATH_INFO: `/v3/projects/d08c78acd1db446bac4352f66e792085/users/c6a8231119f34ef98b8a85303c5c447e/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.565749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:09.583560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08c78acd1db446bac4352f66e792085, user_id=c6a8231119f34ef98b8a85303c5c447e, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.587380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.608116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1123/2184] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:09 2022] PUT /identity/v3/projects/d08c78acd1db446bac4352f66e792085/users/c6a8231119f34ef98b8a85303c5c447e/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:09.613794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a287-9783-4ba2-aac3-166db61170b9 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.614084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a287-9783-4ba2-aac3-166db61170b9 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.614361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a287-9783-4ba2-aac3-166db61170b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.622539 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:09.635450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1a17a287-9783-4ba2-aac3-166db61170b9 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:09.637176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1a17a287-9783-4ba2-aac3-166db61170b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:09.718297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1062/2185] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:09.730351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:09.732495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:09.837901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46868633-9d7c-4a2d-91b3-7907097e42a0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:09.844173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:09.845252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:09.845868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:09.846534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:09.866852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:09.868630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:09.885486 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1124/2186] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:10.752302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b116a5f-6594-4e47-8c33-306f231de705 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:10.752818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b116a5f-6594-4e47-8c33-306f231de705 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:10.752946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b116a5f-6594-4e47-8c33-306f231de705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:10.787916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4b116a5f-6594-4e47-8c33-306f231de705 None None] MFA Rules not processed for user `c6a8231119f34ef98b8a85303c5c447e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:10.789187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4b116a5f-6594-4e47-8c33-306f231de705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:10.886530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1063/2187] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:10.895130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:10.897502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:10.997427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73745b4d-1782-4a33-aa6f-06ce2ea9eed9 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:11.001843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576778e-6491-4e68-993b-3ef6e56e03ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:11.002631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576778e-6491-4e68-993b-3ef6e56e03ec service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:11.002921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576778e-6491-4e68-993b-3ef6e56e03ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:11.003164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576778e-6491-4e68-993b-3ef6e56e03ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:11.020703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576778e-6491-4e68-993b-3ef6e56e03ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:11.023470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576778e-6491-4e68-993b-3ef6e56e03ec service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:11.041715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1125/2188] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:12.562987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:12.563618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:12.563698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:12.587794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] MFA Rules not processed for user `c6a8231119f34ef98b8a85303c5c447e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:12.590647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:12.598297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1064/2189] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:17.640778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b4d27-440e-4fe9-87cf-2edcd64acbeb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:17.641777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b4d27-440e-4fe9-87cf-2edcd64acbeb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:17.644220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b4d27-440e-4fe9-87cf-2edcd64acbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:17.690517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-139b4d27-440e-4fe9-87cf-2edcd64acbeb None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:17.695430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-139b4d27-440e-4fe9-87cf-2edcd64acbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:17.789723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1126/2190] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:17.804047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:17.806488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:17.945837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f51ee7-55e3-45d8-b1bf-86c55f78de43 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:17.951213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:17.951896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:17.952179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:17.952445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:17.963015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:17.964568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:17.974046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1065/2191] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:41:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:41:18.635992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35b23-6fc4-48a3-86ec-7e7fb2700b9a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:18.636277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35b23-6fc4-48a3-86ec-7e7fb2700b9a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:18.636549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35b23-6fc4-48a3-86ec-7e7fb2700b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:18.660464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e2c35b23-6fc4-48a3-86ec-7e7fb2700b9a None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:18.661709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e2c35b23-6fc4-48a3-86ec-7e7fb2700b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:18.675474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1127/2192] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:18.689791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:18.701921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e0d1f-60eb-4c85-98cc-33c20d167d23 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:18.707585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:18.708302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:18.708639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:18.709744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:18.719404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:18.721366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:18.732057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1066/2193] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:41:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:41:19.151903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a91cf-6dcf-41ec-ab5b-fc696bf08a44 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.152352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a91cf-6dcf-41ec-ab5b-fc696bf08a44 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.152771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a91cf-6dcf-41ec-ab5b-fc696bf08a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.183856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-df7a91cf-6dcf-41ec-ab5b-fc696bf08a44 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:19.185436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-df7a91cf-6dcf-41ec-ab5b-fc696bf08a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:19.225538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1128/2194] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:19.240716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:19.250330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:19.255093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffc47b-8b9c-45fd-b8dd-10e46c09cdac service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:19.262505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1be727-ee2a-407e-8ecc-f8cbfca8cf6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:19.263258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1be727-ee2a-407e-8ecc-f8cbfca8cf6f service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.263604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1be727-ee2a-407e-8ecc-f8cbfca8cf6f service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.264002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1be727-ee2a-407e-8ecc-f8cbfca8cf6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.273217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1be727-ee2a-407e-8ecc-f8cbfca8cf6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:19.275462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1be727-ee2a-407e-8ecc-f8cbfca8cf6f service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:19.292007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1067/2195] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:41:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:41:19.445785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a78e9-6dfe-468f-ac65-7a3277b5d205 None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.446039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a78e9-6dfe-468f-ac65-7a3277b5d205 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.446762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a78e9-6dfe-468f-ac65-7a3277b5d205 None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.448424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1129/2196] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:41:19.463941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.464224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.464677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.498667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] MFA Rules not processed for user `53482e2f81364526ae2686f42a7d2483`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:19.506225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:19.573629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a78e9-6dfe-468f-ac65-7a3277b5d205 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:19.575656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ea6a78e9-6dfe-468f-ac65-7a3277b5d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:19.609359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a78e9-6dfe-468f-ac65-7a3277b5d205 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:19.618372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:19.619506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.620180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.620605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] PATH_INFO: `/v3/users/21f622f2e5d8419d9f56a17a48dd89bb` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.622004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f622f2e5d8419d9f56a17a48dd89bb)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:19.624562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:19.648653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:19.671413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1068/2197] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:19.685366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:19.687043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:19.708204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f622f2e5d8419d9f56a17a48dd89bb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:19.709995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f622f2e5d8419d9f56a17a48dd89bb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:19.722178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f622f2e5d8419d9f56a17a48dd89bb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:19.731285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279bcee-3860-4823-aa02-645d5401df18 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:19.738638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:19.739309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.739868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.739983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f622f2e5d8419d9f56a17a48dd89bb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:19.740542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.747521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-290200bc-2278-40d5-8797-cbe7aff9c429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f622f2e5d8419d9f56a17a48dd89bb'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:19.757443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1130/2198] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:19 2022] DELETE /identity/v3/users/21f622f2e5d8419d9f56a17a48dd89bb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:41:19.758537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:19.760511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:19.792861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1069/2199] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:19.865286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:19.865565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:19.865831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:19.889345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:19.890720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:19.978635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1131/2200] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:19.997219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:20.033753 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cce3b2-83c9-43b9-b351-12e1055c1276 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:20.188690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:20.191012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:20.191646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:20.192604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:20.210947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:20.213899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:20.235181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1070/2201] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:41:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:41:20.750512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:20.752341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:20.864680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585070f-bc46-4cc1-8b19-7f69ca53e368 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:20.870472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:20.871207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:20.871719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:20.872082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] PATH_INFO: `/v3/projects/02c82e1afd544a8c8b64d8ddb13028bc` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:20.873422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c82e1afd544a8c8b64d8ddb13028bc)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:20.875694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:20.966932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:20.981749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c82e1afd544a8c8b64d8ddb13028bc', 'request_id': 'req-34682307-384e-413f-9402-e9a5ec171fda'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:20.989967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c82e1afd544a8c8b64d8ddb13028bc', 'request_id': 'req-34682307-384e-413f-9402-e9a5ec171fda'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.003132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c82e1afd544a8c8b64d8ddb13028bc', 'request_id': 'req-34682307-384e-413f-9402-e9a5ec171fda'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.019297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] The token cache is being invalidate because project 02c82e1afd544a8c8b64d8ddb13028bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:21.019735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.020576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.022219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1132/2202] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:20 2022] DELETE /identity/v3/projects/02c82e1afd544a8c8b64d8ddb13028bc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:21.026431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:21.039549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cf030-f898-4201-81f9-68a72b68dc90 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:21.121607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:21.122476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:21.122848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:21.123207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] PATH_INFO: `/v3/users/b1285f18115144d591be2db3b630bb08` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:21.124508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1285f18115144d591be2db3b630bb08)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:21.126549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:21.133159 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:21.196626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1285f18115144d591be2db3b630bb08'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.197776 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1285f18115144d591be2db3b630bb08'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.216702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1285f18115144d591be2db3b630bb08'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.224835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1285f18115144d591be2db3b630bb08'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.225746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a23e3238-0d10-4769-b700-d2c483cc0102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1285f18115144d591be2db3b630bb08'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.251181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1071/2203] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:21 2022] DELETE /identity/v3/users/b1285f18115144d591be2db3b630bb08 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:21.592370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:21.594063 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:21.609497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34682307-384e-413f-9402-e9a5ec171fda admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:21.693770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:21.694421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:21.695067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:21.695067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] PATH_INFO: `/v3/projects/c2b463dd05fb4b5e88d7ca74ea22ab97` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:21.697138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b463dd05fb4b5e88d7ca74ea22ab97)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:21.699158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:21.755801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b463dd05fb4b5e88d7ca74ea22ab97', 'request_id': 'req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.762739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b463dd05fb4b5e88d7ca74ea22ab97', 'request_id': 'req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.774445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b463dd05fb4b5e88d7ca74ea22ab97', 'request_id': 'req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.791178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] The token cache is being invalidate because project c2b463dd05fb4b5e88d7ca74ea22ab97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:21.791465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.792226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-7cb3fc3f-67b4-4e7e-b437-d550c6dc1c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:21.795670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1133/2204] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:21 2022] DELETE /identity/v3/projects/c2b463dd05fb4b5e88d7ca74ea22ab97 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:21.801015 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:21.801293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:21.801722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:21.823537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:21.824888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:21.904670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1072/2205] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:21.912870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:21.913219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:21.913497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:21.935906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:21.937388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:21.946786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1134/2206] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:41:21.951328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:21.962270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dda4f3-cf90-46b0-89d7-bc4f4a300011 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:21.973635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:21.974615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:21.974897 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:21.975169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:21.975930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:41:21.976238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:21.980718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.010747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1073/2207] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:41:22.015305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.029067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819d29d-4cf4-4e80-b9f3-9c71997443f6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.034695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.035360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.035638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.035943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.036662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.038595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.070217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1135/2208] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:22 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:41:22.074634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.087952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8d615-ed87-4a5e-aa4c-96c7ccc64453 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.154005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.154661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.155027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.155213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.155949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.159937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.176886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:22.183512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1074/2209] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:22 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:22.187165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.198600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9514091-affb-46b5-b235-6726d7f0cac4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.203331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.204178 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.204488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.204737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.205455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.207116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.218830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1136/2210] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:22.223409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.236366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730730d3-fe8c-4f5b-aacb-718885f8e1b1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.241801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4491b07-5665-44fb-acb6-80f4a56ca512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.243452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4491b07-5665-44fb-acb6-80f4a56ca512 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.243801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4491b07-5665-44fb-acb6-80f4a56ca512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.244113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4491b07-5665-44fb-acb6-80f4a56ca512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.244843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4491b07-5665-44fb-acb6-80f4a56ca512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.246699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4491b07-5665-44fb-acb6-80f4a56ca512 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.262507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1075/2211] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:22.263929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.275262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1feb61-bbb6-4097-a034-4e54b89d7fb1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.280896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.281491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.281810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.282105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] PATH_INFO: `/v3/projects/ff602177cca44fffa2636a4483bfb3eb/users/d3ab5180e46e4b2795fc0e632bc4d549/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.306213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff602177cca44fffa2636a4483bfb3eb, user_id=d3ab5180e46e4b2795fc0e632bc4d549, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.309345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.314044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:22.327566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1137/2212] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:22 2022] PUT /identity/v3/projects/ff602177cca44fffa2636a4483bfb3eb/users/d3ab5180e46e4b2795fc0e632bc4d549/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:22.340610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.340894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.341342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.372797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] MFA Rules not processed for user `d3ab5180e46e4b2795fc0e632bc4d549`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:22.373702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:22.461509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1076/2213] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:41:22.473692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.475794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:22.541017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5a2ef-15d4-449b-aa6a-802cbb484027 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.544982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d19b5-f051-4a29-bd62-22230d488f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.545607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d19b5-f051-4a29-bd62-22230d488f3c service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.545897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d19b5-f051-4a29-bd62-22230d488f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.546169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d19b5-f051-4a29-bd62-22230d488f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.555798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d19b5-f051-4a29-bd62-22230d488f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.557260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d19b5-f051-4a29-bd62-22230d488f3c service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.566944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1138/2214] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:22.659726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:22.661368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:22.765916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe9662-4063-4287-91d2-71af11d37d2f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:22.771794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:22.772496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.772771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.773114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] PATH_INFO: `/v3/users/c4c08c97c2fb446ba2d363c8a18d4958` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.774432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c08c97c2fb446ba2d363c8a18d4958)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:22.776484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:22.877464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c08c97c2fb446ba2d363c8a18d4958'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:22.878709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c08c97c2fb446ba2d363c8a18d4958'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:22.896232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c08c97c2fb446ba2d363c8a18d4958'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:22.905618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c08c97c2fb446ba2d363c8a18d4958'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:22.906502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c08c97c2fb446ba2d363c8a18d4958'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:22.919003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:22.929278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1077/2215] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:22 2022] DELETE /identity/v3/users/c4c08c97c2fb446ba2d363c8a18d4958 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:41:22.934325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c4fd-8977-4a58-8b67-217197f83313 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:22.934758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c4fd-8977-4a58-8b67-217197f83313 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:22.934908 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c4fd-8977-4a58-8b67-217197f83313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:22.982165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1879c4fd-8977-4a58-8b67-217197f83313 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:22.985227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1879c4fd-8977-4a58-8b67-217197f83313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:23.124312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1139/2216] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:23.135411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:23.137025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:23.242668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917108d-8be7-40c3-91d9-f593292c2499 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:23.246927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:23.247653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:23.248034 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:23.248503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:23.275326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:23.277131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:23.287887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1078/2217] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:23.517978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:23.518676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:23.519162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:23.543582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] MFA Rules not processed for user `1ac7d3acf1204a9f98f9ee14c2b46a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:23.544968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:23.607987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1140/2218] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:23.728171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:23.730238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:23.745360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7498292-d336-4622-8a8c-ca20ca8643e8 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:23.753007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:23.754088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:23.754562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:23.754899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] PATH_INFO: `/v3/projects/c31add21e55d4704b8285a700af04410` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:23.756220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31add21e55d4704b8285a700af04410)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:23.758209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:23.821647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31add21e55d4704b8285a700af04410', 'request_id': 'req-8f34a81a-97af-4bf1-819f-ac281d6298ae'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:23.827609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31add21e55d4704b8285a700af04410', 'request_id': 'req-8f34a81a-97af-4bf1-819f-ac281d6298ae'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:23.838023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31add21e55d4704b8285a700af04410', 'request_id': 'req-8f34a81a-97af-4bf1-819f-ac281d6298ae'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:23.853178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] The token cache is being invalidate because project c31add21e55d4704b8285a700af04410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:23.853531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:23.854255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8f34a81a-97af-4bf1-819f-ac281d6298ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:23.871858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1079/2219] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:23 2022] DELETE /identity/v3/projects/c31add21e55d4704b8285a700af04410 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:33.177044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:33.188471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab27d3-b5f8-4233-a24a-21a9d2f4b835 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:33.239750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:33.248116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:33.248721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:33.248991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:33.249255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:33.251454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:33.319958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:33.321514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:33.332408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1141/2220] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:33.849232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:40.330441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057f9b0-940a-4fe1-86ed-345fbb917e00 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:40.331197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057f9b0-940a-4fe1-86ed-345fbb917e00 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:40.331197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057f9b0-940a-4fe1-86ed-345fbb917e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:40.355230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8057f9b0-940a-4fe1-86ed-345fbb917e00 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:40.357093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8057f9b0-940a-4fe1-86ed-345fbb917e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:40.464017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1080/2221] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:41:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:41:40.505497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:40.507464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:40.625779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efed58-eb63-45f1-8162-d78033287bb6 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:40.631711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:40.632820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:40.633440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:40.634046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:40.644339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:40.645984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:40.658571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1142/2222] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:41:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:41:44.104104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c4662-0772-46cf-95c8-01e3bcb9e067 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:44.105541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c4662-0772-46cf-95c8-01e3bcb9e067 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:44.105991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c4662-0772-46cf-95c8-01e3bcb9e067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:44.130811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-892c4662-0772-46cf-95c8-01e3bcb9e067 None None] MFA Rules not processed for user `c6a8231119f34ef98b8a85303c5c447e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:44.132161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-892c4662-0772-46cf-95c8-01e3bcb9e067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:44.212232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1081/2223] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:44.812272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:44.814686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:44.923700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fabb06-f7ab-4bda-9979-d2af09a54b4f service swift] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:44.929907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:44.930616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:44.931025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:44.931393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] PATH_INFO: `/v3/users/d3ab5180e46e4b2795fc0e632bc4d549` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:44.932668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ab5180e46e4b2795fc0e632bc4d549)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:44.935383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:44.975428 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:45.019507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ab5180e46e4b2795fc0e632bc4d549'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:45.020594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ab5180e46e4b2795fc0e632bc4d549'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:45.046175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ab5180e46e4b2795fc0e632bc4d549'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:45.074916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ab5180e46e4b2795fc0e632bc4d549'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:45.100206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ab5180e46e4b2795fc0e632bc4d549'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:45.125233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1143/2224] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:44 2022] DELETE /identity/v3/users/d3ab5180e46e4b2795fc0e632bc4d549 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:45.126848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:45.127395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:45.127722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:45.155847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:45.157680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:45.302714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1082/2225] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:41:45.313498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:45.315519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:45.492458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4e116-82d4-4832-b431-6bdb514795aa admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:45.503140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5cac-6b01-4cbd-8416-330fd5edb336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:45.504540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5cac-6b01-4cbd-8416-330fd5edb336 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:45.505659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5cac-6b01-4cbd-8416-330fd5edb336 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:45.506197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5cac-6b01-4cbd-8416-330fd5edb336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:45.525933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5cac-6b01-4cbd-8416-330fd5edb336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:45.528585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5cac-6b01-4cbd-8416-330fd5edb336 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:45.566843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1144/2226] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:41:46.225778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:46.227353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:46.254619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b30af6-b5c5-4de7-b977-a8d4dafd5bee None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:46.261017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:46.261946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.262417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.262911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] PATH_INFO: `/v3/projects/ff602177cca44fffa2636a4483bfb3eb` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.264355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff602177cca44fffa2636a4483bfb3eb)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:46.267009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:46.348486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:46.352542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff602177cca44fffa2636a4483bfb3eb', 'request_id': 'req-aa95be6e-d4c0-4312-819a-c744432b5379'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:46.364039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff602177cca44fffa2636a4483bfb3eb', 'request_id': 'req-aa95be6e-d4c0-4312-819a-c744432b5379'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:46.378983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff602177cca44fffa2636a4483bfb3eb', 'request_id': 'req-aa95be6e-d4c0-4312-819a-c744432b5379'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:46.401349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] The token cache is being invalidate because project ff602177cca44fffa2636a4483bfb3eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:46.401645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:46.404374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-aa95be6e-d4c0-4312-819a-c744432b5379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:46.407099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1083/2227] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:46 2022] DELETE /identity/v3/projects/ff602177cca44fffa2636a4483bfb3eb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:46.413175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.413962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.413962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.439362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:46.441215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:46.519271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1145/2228] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:46.530668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.532981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.533799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.579558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:46.581033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:46.594957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1084/2229] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:46.596334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:46.614816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fb139-6c98-4bd8-a904-3868d7683069 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:46.620839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:46.621619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.621901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.622163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.623048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:41:46.624441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:46.626269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:46.642980 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1146/2230] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:41:46.647584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:46.661235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fbf96-d704-42cc-aa02-d0454ce71fa8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:46.670057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:46.670709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.670985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.671255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.672001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:46.674237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:46.719088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1085/2231] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:46 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:46.723930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:46.738021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62965f91-a38f-4a42-8883-6ab704d9a917 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:46.794405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:46.840809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:46.841779 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.842160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.842498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.843734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:46.846447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:46.883324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1147/2232] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:46 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:46.884665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:46.904064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52688bc9-92dd-437f-b8da-1a9f90f39e6c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:46.909722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:46.910379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.910656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.910923 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.911824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:46.913489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:46.956021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1086/2233] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:46.958128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:46.978432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e92be2-550b-4984-9afc-77f38aac1235 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:46.993958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca667456-b738-43d9-9907-2b2c99e865c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:46.995286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667456-b738-43d9-9907-2b2c99e865c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:46.995286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667456-b738-43d9-9907-2b2c99e865c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:46.995530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667456-b738-43d9-9907-2b2c99e865c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:46.996306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca667456-b738-43d9-9907-2b2c99e865c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:47.010773 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca667456-b738-43d9-9907-2b2c99e865c8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:47.046248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1148/2234] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:47.047942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:47.061790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b489a2-08bb-46f6-b6ac-b2ea64502053 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:47.069875 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:47.070960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:47.071318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:47.071637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] PATH_INFO: `/v3/projects/67541b8f463d490590b581458ebb8313/users/61c8a10ed8034756bb3f8994e2eacfa0/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:47.113778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67541b8f463d490590b581458ebb8313, user_id=61c8a10ed8034756bb3f8994e2eacfa0, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:47.124062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:47.148002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1087/2235] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:47 2022] PUT /identity/v3/projects/67541b8f463d490590b581458ebb8313/users/61c8a10ed8034756bb3f8994e2eacfa0/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:47.159878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:47.160188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:47.160625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:47.241550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] MFA Rules not processed for user `61c8a10ed8034756bb3f8994e2eacfa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:47.248491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:47.446275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1149/2236] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:41:47.453987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:47.576614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e81e3-8bc6-421c-981d-e91b83added7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:47.584011 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:47.585158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:47.585533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:47.585936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:47.586998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:47.589334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:47.636742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1088/2237] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:47 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:47.641942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:47.660703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98898d1-3924-42d8-a33a-4313997f0b19 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:47.832952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:47.833978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:47.834360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:47.834734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:47.835696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:47.838006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:47.891679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1150/2238] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:47 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:47.896363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:47.931443 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd70c4d-8971-4964-b330-e40dda6e509e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:47.946286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:47.953294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:47.954380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:47.955152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:47.957614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:47.961370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:47.988081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1089/2239] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:47.994639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.012778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae6168-1537-4c12-901c-d0ebffb36542 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:48.033463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8275813-d404-4096-94f2-4b399f3c623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:48.037529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8275813-d404-4096-94f2-4b399f3c623b admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.037834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8275813-d404-4096-94f2-4b399f3c623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.038071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8275813-d404-4096-94f2-4b399f3c623b admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.038816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8275813-d404-4096-94f2-4b399f3c623b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:48.040544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8275813-d404-4096-94f2-4b399f3c623b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:48.063166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1151/2240] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:48.068761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.090911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f966a-cd5d-4759-b6b9-de46a28eec82 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:48.098935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:48.118969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.118969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.118969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] PATH_INFO: `/v3/projects/a62182aaf2444b5bb96d9f91755b74fb/users/b5ca988d8ba44f3e89af912f5cb6bb95/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.155738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62182aaf2444b5bb96d9f91755b74fb, user_id=b5ca988d8ba44f3e89af912f5cb6bb95, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:48.161268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:48.200969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1090/2241] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:48 2022] PUT /identity/v3/projects/a62182aaf2444b5bb96d9f91755b74fb/users/b5ca988d8ba44f3e89af912f5cb6bb95/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:48.213848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.214107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.214387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.250061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] MFA Rules not processed for user `b5ca988d8ba44f3e89af912f5cb6bb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:48.251602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:48.265719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:48.374702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1152/2242] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:48.379133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.447780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:48.544941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f06a1c-8362-491a-9b8e-5ce07abed16e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:48.560941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:48.562161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.562454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.562786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.564322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:48.572436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:48.626279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1091/2243] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:41:48 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:48.630966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.644527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580397c0-cc5c-4076-8744-f2a75720f005 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:48.754538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:48.755173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.755506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.755813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.756600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:48.759967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:48.807156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.815224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1153/2244] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:41:48 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:41:48.820094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb529b6-244c-4c24-a254-61a9d390f5f5 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:48.827593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:48.831437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.831437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.831437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.831437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:48.831437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:48.861233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.861574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1092/2245] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:48.878989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff38fa7-087f-4e40-8600-6fa96b999ca4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:48.887185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:48.887922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:48.888203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:48.888477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] PATH_INFO: `/v3/projects/4b563d3890be42c38f8bafc7d00dce1d/users/c3830eb5d32247dd911920b88e869179/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:48.915728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b563d3890be42c38f8bafc7d00dce1d, user_id=c3830eb5d32247dd911920b88e869179, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:48.925337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:48.962560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:48.976327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1154/2246] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:48 2022] PUT /identity/v3/projects/4b563d3890be42c38f8bafc7d00dce1d/users/c3830eb5d32247dd911920b88e869179/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:48.995113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6a396-43d4-4100-bd23-b1e1516c93b9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:49.079603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da84af-2d8d-4af7-b1e7-56b2b26901d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:49.080290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da84af-2d8d-4af7-b1e7-56b2b26901d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:49.080571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da84af-2d8d-4af7-b1e7-56b2b26901d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:49.080874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da84af-2d8d-4af7-b1e7-56b2b26901d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:49.081676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da84af-2d8d-4af7-b1e7-56b2b26901d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:49.083291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da84af-2d8d-4af7-b1e7-56b2b26901d9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:49.095503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1093/2247] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:41:49.100456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:49.113637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db48e5-db8b-407c-a148-e41f73e9a22c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:49.119329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:49.120084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:49.120353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:49.120622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] PATH_INFO: `/v3/projects/4b563d3890be42c38f8bafc7d00dce1d/users/c3830eb5d32247dd911920b88e869179/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:49.123023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b563d3890be42c38f8bafc7d00dce1d, user_id=c3830eb5d32247dd911920b88e869179, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:49.135810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:49.147386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1155/2248] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:41:49 2022] PUT /identity/v3/projects/4b563d3890be42c38f8bafc7d00dce1d/users/c3830eb5d32247dd911920b88e869179/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:49.158448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:49.158730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:49.159009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:49.211680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] MFA Rules not processed for user `c3830eb5d32247dd911920b88e869179`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:49.214008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:49.438494 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1094/2249] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:49.450766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:49.454416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:49.566345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f692481-76bb-42df-a027-67ecc3953835 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:49.570804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:49.571525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:49.571831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:49.572107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:49.617712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:49.762113 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:49.764956 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:49.783962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1156/2250] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:51.591622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:51.593228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:51.611930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd71e7-88c6-483c-b52a-57495fc9a360 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:51.615842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:51.617141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:51.617453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:51.617727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:51.636989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:51.710968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:51.714626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:51.727226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1095/2251] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:41:52.657203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:52.667520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a5b58-7b08-4332-87f6-59e5e516b0e3 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:52.672319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cc25d-de58-4063-99ed-81f28c388e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:52.673254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc25d-de58-4063-99ed-81f28c388e58 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:52.673688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc25d-de58-4063-99ed-81f28c388e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:52.674115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc25d-de58-4063-99ed-81f28c388e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:52.690250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cc25d-de58-4063-99ed-81f28c388e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:52.692032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cc25d-de58-4063-99ed-81f28c388e58 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:52.717454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1157/2252] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:56.955953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:57.032799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbeb96-e1bb-4b64-8f00-299c021850a4 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:57.039373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:57.040113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:57.040388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:57.040716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] PATH_INFO: `/v3/users/61c8a10ed8034756bb3f8994e2eacfa0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:57.041929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c8a10ed8034756bb3f8994e2eacfa0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:57.044003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:57.105657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c8a10ed8034756bb3f8994e2eacfa0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:57.106566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c8a10ed8034756bb3f8994e2eacfa0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:57.121267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c8a10ed8034756bb3f8994e2eacfa0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:57.129635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c8a10ed8034756bb3f8994e2eacfa0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:57.130449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c8a10ed8034756bb3f8994e2eacfa0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:57.147830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1096/2253] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:56 2022] DELETE /identity/v3/users/61c8a10ed8034756bb3f8994e2eacfa0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:57.153106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:57.153707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:57.153707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:57.174853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:57.176176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:57.244750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1158/2254] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:57.255038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:57.256489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:57.297706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:57.323960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f7ed6-ce22-45e7-a2ad-5fc8a61c4bce admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:57.328660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:57.329310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:57.329576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:57.329899 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:57.341472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:57.343009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:57.353489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1097/2255] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:58.118495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:58.120270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:58.147225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c71287-5a09-4a23-808a-dec1dac31499 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:58.173004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:58.173671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:58.174156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:58.174585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] PATH_INFO: `/v3/projects/67541b8f463d490590b581458ebb8313` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:58.176210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67541b8f463d490590b581458ebb8313)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:58.178707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:58.244088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67541b8f463d490590b581458ebb8313', 'request_id': 'req-8cb24c93-36d8-4eef-a0ca-df749386cb3f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.252212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67541b8f463d490590b581458ebb8313', 'request_id': 'req-8cb24c93-36d8-4eef-a0ca-df749386cb3f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.265275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67541b8f463d490590b581458ebb8313', 'request_id': 'req-8cb24c93-36d8-4eef-a0ca-df749386cb3f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.279886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] The token cache is being invalidate because project 67541b8f463d490590b581458ebb8313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:58.280606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.281514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.283196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1159/2256] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:58 2022] DELETE /identity/v3/projects/67541b8f463d490590b581458ebb8313 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:58.287574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:58.304645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027781f-e36c-408b-bbe0-bd3807979743 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:58.373787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:58.374844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:58.375168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:58.375514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] PATH_INFO: `/v3/users/b5ca988d8ba44f3e89af912f5cb6bb95` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:58.377050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ca988d8ba44f3e89af912f5cb6bb95)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:58.379689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:58.432218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ca988d8ba44f3e89af912f5cb6bb95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.433035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ca988d8ba44f3e89af912f5cb6bb95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.449913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ca988d8ba44f3e89af912f5cb6bb95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.459169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ca988d8ba44f3e89af912f5cb6bb95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.460031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ca988d8ba44f3e89af912f5cb6bb95'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.477270 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1098/2257] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:58 2022] DELETE /identity/v3/users/b5ca988d8ba44f3e89af912f5cb6bb95 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:58.811297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:58.813632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:58.827590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb24c93-36d8-4eef-a0ca-df749386cb3f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:58.858975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:58.894834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:58.895504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:58.895801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:58.896086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] PATH_INFO: `/v3/projects/a62182aaf2444b5bb96d9f91755b74fb` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:58.897295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62182aaf2444b5bb96d9f91755b74fb)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:58.899215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:58.952660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62182aaf2444b5bb96d9f91755b74fb', 'request_id': 'req-00af5a80-2c7b-492e-8cb1-971916f61575'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.958768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62182aaf2444b5bb96d9f91755b74fb', 'request_id': 'req-00af5a80-2c7b-492e-8cb1-971916f61575'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.968891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62182aaf2444b5bb96d9f91755b74fb', 'request_id': 'req-00af5a80-2c7b-492e-8cb1-971916f61575'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.982024 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] The token cache is being invalidate because project a62182aaf2444b5bb96d9f91755b74fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:58.982315 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.982978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:58.984958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1160/2258] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:58 2022] DELETE /identity/v3/projects/a62182aaf2444b5bb96d9f91755b74fb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:58.988968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:58.990781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:59.005208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0facca-37d6-49b7-9f7e-cdea97045349 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:59.064747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:59.072861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:59.073731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:59.074082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:59.074353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] PATH_INFO: `/v3/users/c3830eb5d32247dd911920b88e869179` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:59.075783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3830eb5d32247dd911920b88e869179)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:59.078302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:59.132404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3830eb5d32247dd911920b88e869179'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.133054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3830eb5d32247dd911920b88e869179'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.147482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3830eb5d32247dd911920b88e869179'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.155698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3830eb5d32247dd911920b88e869179'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.156468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3830eb5d32247dd911920b88e869179'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.173559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1099/2259] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:41:58 2022] DELETE /identity/v3/users/c3830eb5d32247dd911920b88e869179 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:41:59.493656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:59.495142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:59.508249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5a80-2c7b-492e-8cb1-971916f61575 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:59.610103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:59.610935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:59.611948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:59.611948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] PATH_INFO: `/v3/projects/4b563d3890be42c38f8bafc7d00dce1d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:59.613796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b563d3890be42c38f8bafc7d00dce1d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:59.616767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:59.698827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b563d3890be42c38f8bafc7d00dce1d', 'request_id': 'req-cd9fa630-5243-413b-975c-30be9cfa7a1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.706674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b563d3890be42c38f8bafc7d00dce1d', 'request_id': 'req-cd9fa630-5243-413b-975c-30be9cfa7a1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.710569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:41:59.745315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:59.745864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b563d3890be42c38f8bafc7d00dce1d', 'request_id': 'req-cd9fa630-5243-413b-975c-30be9cfa7a1d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.746981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:59.762247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] The token cache is being invalidate because project 4b563d3890be42c38f8bafc7d00dce1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:41:59.762621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.763276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-cd9fa630-5243-413b-975c-30be9cfa7a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:41:59.766806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1161/2260] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:41:59 2022] DELETE /identity/v3/projects/4b563d3890be42c38f8bafc7d00dce1d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:41:59.771471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4419ca-ff43-4f26-9827-913a6cfaeae0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:59.771907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4419ca-ff43-4f26-9827-913a6cfaeae0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:59.772378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4419ca-ff43-4f26-9827-913a6cfaeae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:59.801267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1c4419ca-ff43-4f26-9827-913a6cfaeae0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:59.802502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1c4419ca-ff43-4f26-9827-913a6cfaeae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:59.844662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f97af0-b1ef-4eb6-b7de-a3b277867395 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:59.918200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:59.919911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:59.920195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:59.920520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:59.922548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:59.923796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1162/2261] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:59.932046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:59.932721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:59.933011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:59.942427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:59.945322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:41:59.957137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:41:59.957920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1100/2262] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:41:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:41:59.960707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:41:59.971473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1163/2263] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:41:59.975970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:41:59.989297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff61a9c-3665-453e-930c-801e714581c8 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:41:59.995123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:41:59.995797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:41:59.996209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:41:59.996483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:41:59.997173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:41:59.997471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:41:59.999240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.001278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:00.017725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:00.022409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1101/2264] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:42:00.030535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd18e60-2b90-4c52-bb74-bc1a092b68a6 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:00.037873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:00.039365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.039365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.039365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.039694 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:00.041707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.075319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1164/2265] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:42:00 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:42:00.080247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:00.091102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d92b5-cdde-4635-b94e-306579859dbe admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:00.164111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:00.164761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.165039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.165307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.165978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:00.168166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.200445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1102/2266] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:42:00 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:42:00.200973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:00.213164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5edbc-5d0b-44d5-8b66-a11f1af9ee99 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:00.219048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1ee29-20d2-4e3c-b7e3-2f5f77fecf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:00.219846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1ee29-20d2-4e3c-b7e3-2f5f77fecf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.220175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1ee29-20d2-4e3c-b7e3-2f5f77fecf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.220515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1ee29-20d2-4e3c-b7e3-2f5f77fecf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.221288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1ee29-20d2-4e3c-b7e3-2f5f77fecf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:00.223750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1ee29-20d2-4e3c-b7e3-2f5f77fecf7d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.238581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1165/2267] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:42:00.240748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:00.262598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.262598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.262598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.265948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb82af4-ea3a-4de0-9496-b80e1b2805f2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:00.276902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:00.278163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.278692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.280523 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.282746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:00.285614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.287501 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] MFA Rules not processed for user `4294d6dff5054da7afcb37b8b28f1a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:00.289437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:00.306062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1103/2268] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:42:00.309285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:00.330739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b3b54-f857-48dc-b064-3afbd01315f4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:00.338465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf9640-10b4-4c2f-a7ca-8a9a1ca8d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:00.340533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf9640-10b4-4c2f-a7ca-8a9a1ca8d4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.340973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf9640-10b4-4c2f-a7ca-8a9a1ca8d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.341671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf9640-10b4-4c2f-a7ca-8a9a1ca8d4f8 admin admin] PATH_INFO: `/v3/projects/b71b8251e4f744c085fb97353a2301ca/users/876cda5d1e2443e181611b492002ba4f/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.365608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf9640-10b4-4c2f-a7ca-8a9a1ca8d4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71b8251e4f744c085fb97353a2301ca, user_id=876cda5d1e2443e181611b492002ba4f, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:00.368355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf9640-10b4-4c2f-a7ca-8a9a1ca8d4f8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.396046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1104/2269] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:42:00 2022] PUT /identity/v3/projects/b71b8251e4f744c085fb97353a2301ca/users/876cda5d1e2443e181611b492002ba4f/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:00.399241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1166/2270] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:42:00.400940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495a1a6-76f9-433e-ac06-a32c98b19c2f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.401226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495a1a6-76f9-433e-ac06-a32c98b19c2f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.401502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495a1a6-76f9-433e-ac06-a32c98b19c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.438304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5495a1a6-76f9-433e-ac06-a32c98b19c2f None None] MFA Rules not processed for user `876cda5d1e2443e181611b492002ba4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:00.439573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5495a1a6-76f9-433e-ac06-a32c98b19c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:00.518807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1105/2271] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:00.532138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:00.534009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:00.606293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d42289-87d2-490c-b9d2-16cda1dd09f1 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:00.613901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:00.681505 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335db2d6-1a52-410a-8ce5-a70f4ffa253f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:00.682644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335db2d6-1a52-410a-8ce5-a70f4ffa253f service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:00.682814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335db2d6-1a52-410a-8ce5-a70f4ffa253f service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:00.683186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335db2d6-1a52-410a-8ce5-a70f4ffa253f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:00.700036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335db2d6-1a52-410a-8ce5-a70f4ffa253f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:00.702203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335db2d6-1a52-410a-8ce5-a70f4ffa253f service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:00.716115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1167/2272] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:02.443048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780832ec-309c-4b85-b733-083e380305b2 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:02.443641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780832ec-309c-4b85-b733-083e380305b2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:02.443641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780832ec-309c-4b85-b733-083e380305b2 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:02.445072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1106/2273] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:42:02.451666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42e34b-152b-4dfd-b0a2-bd070ae4c794 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:02.452375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42e34b-152b-4dfd-b0a2-bd070ae4c794 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:02.452375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42e34b-152b-4dfd-b0a2-bd070ae4c794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:02.475555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fc42e34b-152b-4dfd-b0a2-bd070ae4c794 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:02.476920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fc42e34b-152b-4dfd-b0a2-bd070ae4c794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:02.587026 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1168/2274] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:02.597904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780832ec-309c-4b85-b733-083e380305b2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:02.599260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-780832ec-309c-4b85-b733-083e380305b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:02.646385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:02.665817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780832ec-309c-4b85-b733-083e380305b2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:02.669616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b891971-bf55-4dd4-b117-d20c60ed7c62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:02.670249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891971-bf55-4dd4-b117-d20c60ed7c62 service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:02.670525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891971-bf55-4dd4-b117-d20c60ed7c62 service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:02.670962 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b891971-bf55-4dd4-b117-d20c60ed7c62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:02.682551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891971-bf55-4dd4-b117-d20c60ed7c62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:02.684028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b891971-bf55-4dd4-b117-d20c60ed7c62 service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:02.695508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1107/2275] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:02.771897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476049-2f4c-4afd-a839-bd8f8560157f None None] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:02.772184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476049-2f4c-4afd-a839-bd8f8560157f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:02.772457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba476049-2f4c-4afd-a839-bd8f8560157f None None] PATH_INFO: `/` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:02.774004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1169/2276] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:42:02.779452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0fb3e-4fa8-46dc-a621-1eb4d8cac067 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:02.779732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0fb3e-4fa8-46dc-a621-1eb4d8cac067 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:02.780226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0fb3e-4fa8-46dc-a621-1eb4d8cac067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:02.803902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-64c0fb3e-4fa8-46dc-a621-1eb4d8cac067 None None] MFA Rules not processed for user `d85bf2b530ce423d8f1c010b5044dba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:02.805129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-64c0fb3e-4fa8-46dc-a621-1eb4d8cac067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:02.859347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1108/2277] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:02.871044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476049-2f4c-4afd-a839-bd8f8560157f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:02.872888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ba476049-2f4c-4afd-a839-bd8f8560157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:02.887492 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba476049-2f4c-4afd-a839-bd8f8560157f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:02.892650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d46dea-aa01-41fe-a6e8-a4c8dd26cdea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:02.893311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d46dea-aa01-41fe-a6e8-a4c8dd26cdea service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:02.893585 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d46dea-aa01-41fe-a6e8-a4c8dd26cdea service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:02.893851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d46dea-aa01-41fe-a6e8-a4c8dd26cdea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:02.901490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d46dea-aa01-41fe-a6e8-a4c8dd26cdea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:02.902924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d46dea-aa01-41fe-a6e8-a4c8dd26cdea service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:02.911271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1170/2278] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:03.004111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc94f0-3cd8-4cca-ac11-c0b64ce1cc56 None None] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:03.004425 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc94f0-3cd8-4cca-ac11-c0b64ce1cc56 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:03.004700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc94f0-3cd8-4cca-ac11-c0b64ce1cc56 None None] PATH_INFO: `/` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:03.006139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1109/2279] 10.222.0.45 () {54 vars in 921 bytes} [Mon Aug 1 16:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:42:03.011588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:03.011909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:03.012208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:03.034155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] MFA Rules not processed for user `dd1da21c00d8494587e3ede50a5a365b`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:03.035383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:03.047750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1171/2280] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Aug 1 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:42:03.058308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc94f0-3cd8-4cca-ac11-c0b64ce1cc56 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:03.070458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc94f0-3cd8-4cca-ac11-c0b64ce1cc56 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:03.074321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaf9bf-3717-4c91-8305-483e3e130e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:03.074957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaf9bf-3717-4c91-8305-483e3e130e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:03.075228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaf9bf-3717-4c91-8305-483e3e130e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:03.075494 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaf9bf-3717-4c91-8305-483e3e130e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:03.088535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaf9bf-3717-4c91-8305-483e3e130e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:03.090076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaf9bf-3717-4c91-8305-483e3e130e6f service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:03.103736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1110/2281] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:03.422163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:03.491702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a43904-ff35-46b3-bf3f-e1b147474f60 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:03.496564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:03.497250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:03.497565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:03.497896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] PATH_INFO: `/v3/users/876cda5d1e2443e181611b492002ba4f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:03.499189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=876cda5d1e2443e181611b492002ba4f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:03.501359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:03.535135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:03.557507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876cda5d1e2443e181611b492002ba4f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:03.558173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876cda5d1e2443e181611b492002ba4f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:03.567839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876cda5d1e2443e181611b492002ba4f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:03.582984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876cda5d1e2443e181611b492002ba4f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:03.593039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876cda5d1e2443e181611b492002ba4f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:03.603813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1172/2282] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:42:03 2022] DELETE /identity/v3/users/876cda5d1e2443e181611b492002ba4f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:03.606190 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:03.606655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:03.607349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:03.629562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:03.631603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:03.744749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1111/2283] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:03.751970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:03.753999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:03.833521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c6b06-156d-4d2b-90c5-eaa6918b6f59 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:03.839687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ff2b0-4320-4a11-9043-2fb2b51e467e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:03.840567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ff2b0-4320-4a11-9043-2fb2b51e467e service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:03.841690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ff2b0-4320-4a11-9043-2fb2b51e467e service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:03.842042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ff2b0-4320-4a11-9043-2fb2b51e467e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:03.855058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ff2b0-4320-4a11-9043-2fb2b51e467e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:03.856820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ff2b0-4320-4a11-9043-2fb2b51e467e service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:03.871114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1173/2284] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:04.197530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.199113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:04.212346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d194ef-0203-45ae-96c5-033039431176 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.217408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.218332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.218608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.218881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] PATH_INFO: `/v3/projects/b71b8251e4f744c085fb97353a2301ca` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.220191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71b8251e4f744c085fb97353a2301ca)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.222158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.275366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71b8251e4f744c085fb97353a2301ca', 'request_id': 'req-eb2660c6-b626-455b-a50f-246f16f4e033'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:04.281786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71b8251e4f744c085fb97353a2301ca', 'request_id': 'req-eb2660c6-b626-455b-a50f-246f16f4e033'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:04.292290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71b8251e4f744c085fb97353a2301ca', 'request_id': 'req-eb2660c6-b626-455b-a50f-246f16f4e033'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:04.301898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:04.306701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] The token cache is being invalidate because project b71b8251e4f744c085fb97353a2301ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:42:04.307361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:04.308249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-eb2660c6-b626-455b-a50f-246f16f4e033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:04.310161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1112/2285] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:42:04 2022] DELETE /identity/v3/projects/b71b8251e4f744c085fb97353a2301ca => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:04.316189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.316485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.316858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.339201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:04.340475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:04.408526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1174/2286] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:04.482374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.482613 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.482901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.505133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:04.506360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:04.516294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1113/2287] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:04.525177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.536725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5954c-5ef2-47d8-8b90-bbc2a5ac0045 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.542725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.543837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.544217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.544727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.546507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:42:04.547041 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.550108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.568660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1175/2288] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:42:04.572095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.583119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c28b2-fa61-4ae7-8810-f4c3b8711789 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.588618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.589253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.589612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.590004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.590792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.593172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.624928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1114/2289] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:42:04 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:42:04.630526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.641774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024a4f-9a8a-480e-a677-540d058758f5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.692804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:04.731728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.732460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.732755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.733034 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.733723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.736903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.763559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1176/2290] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:42:04 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:42:04.767681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.778566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991f7de-acbd-4572-a102-17e4d1e909c1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.783411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.784079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.784363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.784627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.785360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.786979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.798661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1115/2291] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:42:04.802780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.814368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b2458-4d18-4f49-893f-309aa3e19bdf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.819316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c19d18-9da3-450a-9021-5b07816eac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.819937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19d18-9da3-450a-9021-5b07816eac93 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.820230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19d18-9da3-450a-9021-5b07816eac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.821128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c19d18-9da3-450a-9021-5b07816eac93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.822043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19d18-9da3-450a-9021-5b07816eac93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.823804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c19d18-9da3-450a-9021-5b07816eac93 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.835880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1177/2292] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:42:04.840318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:04.851530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe889101-f4f3-420b-b12c-5a43e8922b77 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:04.856417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:04.857167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.857516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.857855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] PATH_INFO: `/v3/projects/9619d6bd228449899b8b067432ba5ed3/users/018f94ceb1f543028114b83d8f935c31/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.875312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9619d6bd228449899b8b067432ba5ed3, user_id=018f94ceb1f543028114b83d8f935c31, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:04.878139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:04.893522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1116/2293] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:42:04 2022] PUT /identity/v3/projects/9619d6bd228449899b8b067432ba5ed3/users/018f94ceb1f543028114b83d8f935c31/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:04.903910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d3344-06bc-461d-9507-f691db57d11f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:04.904267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d3344-06bc-461d-9507-f691db57d11f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:04.904631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d3344-06bc-461d-9507-f691db57d11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:04.941126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-232d3344-06bc-461d-9507-f691db57d11f None None] MFA Rules not processed for user `018f94ceb1f543028114b83d8f935c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:04.942625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-232d3344-06bc-461d-9507-f691db57d11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:05.116454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1178/2294] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:05.144764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:05.146986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:05.237029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5ec2-19e6-4bd0-befe-beb2b1f05191 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:05.244330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:05.245374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:05.245828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:05.246254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:05.257570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:05.259275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:05.295818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1117/2295] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:13.327979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f1a52-a4e5-4619-be4e-93820009e28f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:13.328598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f1a52-a4e5-4619-be4e-93820009e28f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:13.328598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f1a52-a4e5-4619-be4e-93820009e28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:13.353778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d76f1a52-a4e5-4619-be4e-93820009e28f None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:13.355605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d76f1a52-a4e5-4619-be4e-93820009e28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:13.446721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1179/2296] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:13.462229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:13.464001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:13.585619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f075e-8bd4-4dbd-888a-33cc960eae57 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:13.591792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:13.592701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:13.592986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:13.593317 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:13.603612 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:13.605304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:13.614935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1118/2297] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:42:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:42:14.417383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:14.417991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:14.417991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:14.447655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:14.449630 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:14.463157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1180/2298] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:14.476453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:14.488012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593ded5-4dfd-45e5-ba9e-fe06ca83d542 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:14.493669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8aff5-d74a-4d5f-a5fb-b5ce79cb0d6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:14.494293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8aff5-d74a-4d5f-a5fb-b5ce79cb0d6f service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:14.494547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8aff5-d74a-4d5f-a5fb-b5ce79cb0d6f service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:14.494841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8aff5-d74a-4d5f-a5fb-b5ce79cb0d6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:14.507637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8aff5-d74a-4d5f-a5fb-b5ce79cb0d6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:14.510068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8aff5-d74a-4d5f-a5fb-b5ce79cb0d6f service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:14.523099 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1119/2299] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:42:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:42:17.121319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:17.213542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:17.271322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d0418-2f59-48d6-a4e7-7142d0f72a60 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:17.281054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:17.282518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:17.283164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:17.283605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] PATH_INFO: `/v3/users/018f94ceb1f543028114b83d8f935c31` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:17.287193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018f94ceb1f543028114b83d8f935c31)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:17.295869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:17.402682 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018f94ceb1f543028114b83d8f935c31'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:17.403320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018f94ceb1f543028114b83d8f935c31'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:17.421930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018f94ceb1f543028114b83d8f935c31'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:17.449336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018f94ceb1f543028114b83d8f935c31'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:17.484844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018f94ceb1f543028114b83d8f935c31'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:17.495812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1181/2300] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:42:17 2022] DELETE /identity/v3/users/018f94ceb1f543028114b83d8f935c31 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:17.498364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:17.499180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:17.499371 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:17.550440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:17.552261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:17.591249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:17.759921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1120/2301] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:17.768841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:17.782509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:17.914817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0e6f-7e12-4c85-b7a6-388ac33ed2d3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:17.918603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c22fe-01d8-41ee-ba15-44a948a3ba54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:17.919446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c22fe-01d8-41ee-ba15-44a948a3ba54 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:17.919726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c22fe-01d8-41ee-ba15-44a948a3ba54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:17.920290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c22fe-01d8-41ee-ba15-44a948a3ba54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:17.932515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c22fe-01d8-41ee-ba15-44a948a3ba54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:17.935862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c22fe-01d8-41ee-ba15-44a948a3ba54 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:17.949436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1182/2302] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:42:18.459946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:18.462337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:18.486413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bdaf2-25b3-4b5c-925d-d9a2d721dd14 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:18.502574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:18.504098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:18.505085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:18.506323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] PATH_INFO: `/v3/projects/9619d6bd228449899b8b067432ba5ed3` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:18.514584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9619d6bd228449899b8b067432ba5ed3)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:18.517056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:18.663306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9619d6bd228449899b8b067432ba5ed3', 'request_id': 'req-655f0b62-3c39-482c-a489-0aaf10ddd019'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:18.683012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9619d6bd228449899b8b067432ba5ed3', 'request_id': 'req-655f0b62-3c39-482c-a489-0aaf10ddd019'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:18.698770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9619d6bd228449899b8b067432ba5ed3', 'request_id': 'req-655f0b62-3c39-482c-a489-0aaf10ddd019'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:18.720091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] The token cache is being invalidate because project 9619d6bd228449899b8b067432ba5ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:42:18.720553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:18.721499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:18.724097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1121/2303] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:42:18 2022] DELETE /identity/v3/projects/9619d6bd228449899b8b067432ba5ed3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:27.944685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:28.715077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:32.083360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:32.084141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:32.084141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:32.105342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:32.106582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:32.184524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1183/2304] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:32.201679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:32.203515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:32.288365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f0b62-3c39-482c-a489-0aaf10ddd019 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:32.296019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:32.296788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:32.297153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:32.297509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:32.307666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:32.309364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:32.319743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1122/2305] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:42:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:42:34.441036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:34.442933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:34.545196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fdc8a-87c2-4457-9d9e-1e7e357a3a4d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:34.551481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:34.552354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:34.552721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:34.553082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] PATH_INFO: `/v3/users/c6a8231119f34ef98b8a85303c5c447e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:34.554599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a8231119f34ef98b8a85303c5c447e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:34.556790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:34.620103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a8231119f34ef98b8a85303c5c447e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:34.620868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a8231119f34ef98b8a85303c5c447e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:34.631206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a8231119f34ef98b8a85303c5c447e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:34.647719 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a8231119f34ef98b8a85303c5c447e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:34.656864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-db3d7898-6182-467c-9b24-58ab0910bd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a8231119f34ef98b8a85303c5c447e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:34.664540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1184/2306] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:42:34 2022] DELETE /identity/v3/users/c6a8231119f34ef98b8a85303c5c447e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:34.958453 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:34.981802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06b42-bdaa-47dd-8428-0effe2827f86 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.066621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.067842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.068399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.068905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] PATH_INFO: `/v3/projects/d08c78acd1db446bac4352f66e792085` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.071630 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08c78acd1db446bac4352f66e792085)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.073719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.151464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08c78acd1db446bac4352f66e792085', 'request_id': 'req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:35.162088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08c78acd1db446bac4352f66e792085', 'request_id': 'req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:35.174815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08c78acd1db446bac4352f66e792085', 'request_id': 'req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:35.195485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] The token cache is being invalidate because project d08c78acd1db446bac4352f66e792085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:42:35.195843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:35.196536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a54fe49a-f2a6-4cde-a8a6-b501e7c9415c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:42:35.198145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1123/2307] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:42:34 2022] DELETE /identity/v3/projects/d08c78acd1db446bac4352f66e792085 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:35.203749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.204069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.204321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.227858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:35.229336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:35.254361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:35.304861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1185/2308] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:35.314228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.314570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.314876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.338472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:35.339818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:35.350199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1124/2309] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:35.355402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.368387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50950f6-46ef-4550-a9a2-42b00b6e897f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.374240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.374909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.375194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.375464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.376278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:42:35.376581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.378240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.393778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1186/2310] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:42:35.399024 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.411536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1873-879c-440f-a5ae-33b5e3bc4f42 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.417725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.418429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.418708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.418980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.419669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.421863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.436094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:35.463429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1125/2311] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:42:35 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:42:35.466650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.485303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2a66c-4185-43c2-9380-2f5c4211507d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.566006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.566640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.566931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.567185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.567975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.570107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.596391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1187/2312] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:42:35 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:42:35.601943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.614757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaa5b2-5d6b-40aa-9089-c87890b93fba admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.620209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.621216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.621533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.621895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.622759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.624751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.637646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1126/2313] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:42:35.642816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.661562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e3c11-3d3b-4e67-b5f4-f1b77d7b2dea admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.670748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6e02b-0c99-4ea2-96c2-6721ece6925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.671563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6e02b-0c99-4ea2-96c2-6721ece6925c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.672078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6e02b-0c99-4ea2-96c2-6721ece6925c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.672655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6e02b-0c99-4ea2-96c2-6721ece6925c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.674016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6e02b-0c99-4ea2-96c2-6721ece6925c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.676402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6e02b-0c99-4ea2-96c2-6721ece6925c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.696850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1188/2314] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:42:35.701818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.713450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298b63-6d4a-4f10-89ab-9b14201a480a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:35.721710 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:35.722867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.723147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.723987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] PATH_INFO: `/v3/projects/150de159cf784247be229fa30f6b8d8d/users/9c093e19e9124acca1f4e8611a412531/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.742114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150de159cf784247be229fa30f6b8d8d, user_id=9c093e19e9124acca1f4e8611a412531, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:35.745243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:35.769992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1127/2315] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:42:35 2022] PUT /identity/v3/projects/150de159cf784247be229fa30f6b8d8d/users/9c093e19e9124acca1f4e8611a412531/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:42:35.788226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4557a7-2b4b-4232-82ed-fca48f6c4b16 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:35.788723 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4557a7-2b4b-4232-82ed-fca48f6c4b16 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:35.789175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4557a7-2b4b-4232-82ed-fca48f6c4b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:35.820973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ab4557a7-2b4b-4232-82ed-fca48f6c4b16 None None] MFA Rules not processed for user `9c093e19e9124acca1f4e8611a412531`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:35.822198 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ab4557a7-2b4b-4232-82ed-fca48f6c4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:35.927829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1189/2316] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:35.945070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:35.947226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:36.034058 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a86dd1-011a-487a-9306-acb862021815 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:36.038492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:36.039225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:36.039550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:36.039859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:36.052764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:36.054445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:36.067168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1128/2317] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:42:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:42:40.882093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:40.882722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:40.882722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:40.907007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:42:40.908438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:40.928539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:40.986558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1190/2318] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 16:42:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:42:41.012825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:42:41.014529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:42:41.101355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7af13d-b0dc-4702-a4fb-ca7d6720c5b3 service glance] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:42:41.110133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c79c-821c-4e7a-bdc0-8898ade8bb38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:42:41.111054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c79c-821c-4e7a-bdc0-8898ade8bb38 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:42:41.111347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c79c-821c-4e7a-bdc0-8898ade8bb38 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:42:41.111603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c79c-821c-4e7a-bdc0-8898ade8bb38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:42:41.122727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1c79c-821c-4e7a-bdc0-8898ade8bb38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:42:41.124763 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1c79c-821c-4e7a-bdc0-8898ade8bb38 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:42:41.148184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1129/2319] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:42:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:42:50.978704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=16 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:42:51.131031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=76,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:06.191960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:06.262551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3779e7-849d-42b3-945c-8ee176a52179 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:06.267419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:06.268099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:06.268376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:06.268702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] PATH_INFO: `/v3/users/9c093e19e9124acca1f4e8611a412531` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:06.269886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c093e19e9124acca1f4e8611a412531)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:06.271863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:06.331025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c093e19e9124acca1f4e8611a412531'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.331639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c093e19e9124acca1f4e8611a412531'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.341753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c093e19e9124acca1f4e8611a412531'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.354874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c093e19e9124acca1f4e8611a412531'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.363099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c093e19e9124acca1f4e8611a412531'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.374179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1191/2320] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:43:06 2022] DELETE /identity/v3/users/9c093e19e9124acca1f4e8611a412531 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:06.378871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:06.379432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:06.379432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:06.405553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:06.406751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:06.471898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1130/2321] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:06.483449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:06.484987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:06.549030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4494d2c-e504-45c4-9462-a7f6339a75c6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:06.553537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a972b7c-9dbd-4f37-9af6-1c1fcf16d8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:06.554172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a972b7c-9dbd-4f37-9af6-1c1fcf16d8fb service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:06.554450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a972b7c-9dbd-4f37-9af6-1c1fcf16d8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:06.554710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a972b7c-9dbd-4f37-9af6-1c1fcf16d8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:06.565030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a972b7c-9dbd-4f37-9af6-1c1fcf16d8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:06.566700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a972b7c-9dbd-4f37-9af6-1c1fcf16d8fb service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:06.576785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1192/2322] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:43:06.870206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:06.872152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:06.885474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510059f-c441-477c-bb34-5c67889a75c1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:06.891790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:06.892517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:06.892726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:06.893093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] PATH_INFO: `/v3/projects/150de159cf784247be229fa30f6b8d8d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:06.894205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=150de159cf784247be229fa30f6b8d8d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:06.897495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:06.954803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150de159cf784247be229fa30f6b8d8d', 'request_id': 'req-7724452e-9aab-4237-bd8a-642542b2e381'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.960289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150de159cf784247be229fa30f6b8d8d', 'request_id': 'req-7724452e-9aab-4237-bd8a-642542b2e381'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.969330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150de159cf784247be229fa30f6b8d8d', 'request_id': 'req-7724452e-9aab-4237-bd8a-642542b2e381'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.983663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] The token cache is being invalidate because project 150de159cf784247be229fa30f6b8d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:43:06.984115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.984805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7724452e-9aab-4237-bd8a-642542b2e381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:06.986810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1131/2323] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:43:06 2022] DELETE /identity/v3/projects/150de159cf784247be229fa30f6b8d8d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:06.992455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:06.992742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:06.993031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.015941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:07.017180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:07.036271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:07.087783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1193/2324] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:07.098204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.098446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.098750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.122382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:07.123507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:07.132233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1132/2325] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:07.136857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.147177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32dcfd-7a14-4b91-84ed-6f36c4330b46 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.152664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.153288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.153563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.153836 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.154544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:43:07.154835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.156481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.168740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1194/2326] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:43:07.173732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.184322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a12aea-ef26-456b-ad38-d062ac6704bf None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.189530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.190149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.190413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.190692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.191361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.193311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.224907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1133/2327] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:43:07 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:43:07.229971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.241601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790965-3f83-4542-aacc-ea547f532d59 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.307515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.308187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.308493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.308817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.309480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.311551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.335828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1195/2328] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:43:07 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:43:07.339490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.351352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f948e-2aec-4a5d-a2d1-a4cd5ed48f62 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.357553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.358201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.358480 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.358750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.359454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.361242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.372963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1134/2329] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:43:07.377039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.387798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b046f1-76bb-48c5-90b4-cabd1f07c853 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.392563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc74f0-1f3d-492b-ac30-961cdf633e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.393182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc74f0-1f3d-492b-ac30-961cdf633e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.393456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc74f0-1f3d-492b-ac30-961cdf633e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.393756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc74f0-1f3d-492b-ac30-961cdf633e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.394462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc74f0-1f3d-492b-ac30-961cdf633e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.396097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc74f0-1f3d-492b-ac30-961cdf633e07 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.407815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1196/2330] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:43:07.412138 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.423311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a0c12-e6f8-4c83-8ae8-13a68121ed05 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.428436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.429088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.429367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.429641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] PATH_INFO: `/v3/projects/c86fcc5e23854dfba97058deac8211d0/users/22cfd17a6f434deca7ef77253ae3ed2f/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.446028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86fcc5e23854dfba97058deac8211d0, user_id=22cfd17a6f434deca7ef77253ae3ed2f, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.448625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.464843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1135/2331] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:43:07 2022] PUT /identity/v3/projects/c86fcc5e23854dfba97058deac8211d0/users/22cfd17a6f434deca7ef77253ae3ed2f/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:43:07.470612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571f318-3215-44cc-9b04-0a7b63d44080 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.470894 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571f318-3215-44cc-9b04-0a7b63d44080 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.471170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571f318-3215-44cc-9b04-0a7b63d44080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.491742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4571f318-3215-44cc-9b04-0a7b63d44080 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:07.493027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4571f318-3215-44cc-9b04-0a7b63d44080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:07.564686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1197/2332] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:43:07.574883 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:07.577166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:07.600725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:07.640121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471efc58-d7c8-4ce0-b5c1-eb60b0cb37d8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:07.644266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:07.644901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:07.645163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:07.645435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:07.656003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:07.657454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:07.667481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1136/2333] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:43:12.220813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.221371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.221371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.254868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] MFA Rules not processed for user `22cfd17a6f434deca7ef77253ae3ed2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:12.256262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:12.281673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:12.387139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1198/2334] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:12.391963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:12.421795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ece97-7469-4e1f-86d4-5b55a43ac648 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:12.434463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:12.437171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.437694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.440297 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.441306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:12.460478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:12.532242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1137/2335] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:43:12 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:43:12.533594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:12.552530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f4d8-d73d-4715-945b-da8eefaabacc None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:12.654517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:12.655004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.655280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.655552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.656278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:12.659400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:12.698745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1199/2336] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:43:12 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:43:12.700563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:12.712225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55916820-8938-43b6-a6bd-42f9af83df3b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:12.718579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:12.719275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.719943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.720503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.720949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:12.722590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:12.734745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1138/2337] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:43:12.739106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:12.750842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bd77e-c246-4398-996b-a415e6eeba9f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:12.756939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:12.757675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.757953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.758224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] PATH_INFO: `/v3/projects/d5d77d23bd50495798ff519f623d1fb4/users/78e852884c584197a68547531bcaa2b5/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.785267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d77d23bd50495798ff519f623d1fb4, user_id=78e852884c584197a68547531bcaa2b5, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:12.788158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:12.806605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1200/2338] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:43:12 2022] PUT /identity/v3/projects/d5d77d23bd50495798ff519f623d1fb4/users/78e852884c584197a68547531bcaa2b5/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:12.810574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:12.820918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79a0cc-6e37-4131-9286-d4f6dbcf1989 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:12.942588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae947b9e-9800-4694-8aaa-4a1d98a28004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:12.948178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae947b9e-9800-4694-8aaa-4a1d98a28004 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.949098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae947b9e-9800-4694-8aaa-4a1d98a28004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.949098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae947b9e-9800-4694-8aaa-4a1d98a28004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.949687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae947b9e-9800-4694-8aaa-4a1d98a28004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:12.954273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae947b9e-9800-4694-8aaa-4a1d98a28004 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:12.969057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1139/2339] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:43:12.974677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:12.987408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b279a5-15b1-4593-9e55-50e5848fa80d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:12.992703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7012102-a039-439b-9b1a-90248bc790d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:12.993395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7012102-a039-439b-9b1a-90248bc790d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:12.993670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7012102-a039-439b-9b1a-90248bc790d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:12.993942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7012102-a039-439b-9b1a-90248bc790d6 admin admin] PATH_INFO: `/v3/projects/d5d77d23bd50495798ff519f623d1fb4/users/78e852884c584197a68547531bcaa2b5/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:12.996370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7012102-a039-439b-9b1a-90248bc790d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d77d23bd50495798ff519f623d1fb4, user_id=78e852884c584197a68547531bcaa2b5, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:12.999226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7012102-a039-439b-9b1a-90248bc790d6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:13.024447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1201/2340] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:43:12 2022] PUT /identity/v3/projects/d5d77d23bd50495798ff519f623d1fb4/users/78e852884c584197a68547531bcaa2b5/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:17.111687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:17.112295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:17.112295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:17.147129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] MFA Rules not processed for user `1ac7d3acf1204a9f98f9ee14c2b46a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:17.157197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:17.307804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1140/2341] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:19.035472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1299-cf46-4044-9b94-046013b8dafd None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:19.036090 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1299-cf46-4044-9b94-046013b8dafd None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:19.036297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd1299-cf46-4044-9b94-046013b8dafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:19.073404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-4fbd1299-cf46-4044-9b94-046013b8dafd None None] MFA Rules not processed for user `78e852884c584197a68547531bcaa2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:19.075013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4fbd1299-cf46-4044-9b94-046013b8dafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:19.166025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:19.172919 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1202/2342] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:19.187552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:19.189107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:19.240810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:19.292214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0740d5-8cd1-4741-b08e-c2f9a32e1c9f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:19.298765 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:19.299418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:19.299698 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:19.300004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:19.382879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:19.387733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:19.402778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1141/2343] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:43:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:43:29.177801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:29.400568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=42 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:36.867052 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa914c77-c5c7-4789-bcc8-038173430c89 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:36.867578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa914c77-c5c7-4789-bcc8-038173430c89 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:36.867578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa914c77-c5c7-4789-bcc8-038173430c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:36.888825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-fa914c77-c5c7-4789-bcc8-038173430c89 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:36.890066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-fa914c77-c5c7-4789-bcc8-038173430c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:36.951931 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1203/2344] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:36.964728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:36.966367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:37.050334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42eaf7-9d78-4241-9f50-be35f274218b service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:37.055926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:37.056817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:37.057091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:37.057593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:37.066491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:37.068223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:37.076592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1142/2345] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:43:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:43:38.146466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:38.147018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:38.147018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:38.174550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] MFA Rules not processed for user `22cfd17a6f434deca7ef77253ae3ed2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:38.175705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:38.183360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1204/2346] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:43:47.073737 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=32 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:47.849784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:47.852298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:47.968521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399c197-e988-4944-a57c-ffd961c0ba6d service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:47.990238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:47.991657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:47.992067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:47.992438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] PATH_INFO: `/v3/users/23a95d9f48c64b0fbb8879d361bc177d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:47.994087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a95d9f48c64b0fbb8879d361bc177d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:47.996725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:48.102098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a95d9f48c64b0fbb8879d361bc177d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.105081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a95d9f48c64b0fbb8879d361bc177d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.129068 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a95d9f48c64b0fbb8879d361bc177d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.141910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a95d9f48c64b0fbb8879d361bc177d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.142831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a95d9f48c64b0fbb8879d361bc177d'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.166224 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=28 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:48.175308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1143/2347] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:43:47 2022] DELETE /identity/v3/users/23a95d9f48c64b0fbb8879d361bc177d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:48.549042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:48.563170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a9bdd-dc04-4db9-81fd-f7f016d508d7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:48.626483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:48.627100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:48.627391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:48.627646 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] PATH_INFO: `/v3/projects/019734069da44b2d88561549bd07544d` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:48.628834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=019734069da44b2d88561549bd07544d)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:48.630904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:48.687362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019734069da44b2d88561549bd07544d', 'request_id': 'req-40987d32-67c4-4d85-9b4e-be14df0f6e98'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.695121 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019734069da44b2d88561549bd07544d', 'request_id': 'req-40987d32-67c4-4d85-9b4e-be14df0f6e98'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.706726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019734069da44b2d88561549bd07544d', 'request_id': 'req-40987d32-67c4-4d85-9b4e-be14df0f6e98'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.729661 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] The token cache is being invalidate because project 019734069da44b2d88561549bd07544d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:43:48.730112 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.731151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-40987d32-67c4-4d85-9b4e-be14df0f6e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.739438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1205/2348] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:43:48 2022] DELETE /identity/v3/projects/019734069da44b2d88561549bd07544d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:48.740850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:48.742444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:48.758002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcafab-a74e-458b-ba6a-0fd043d06153 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:48.837345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:48.838295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:48.838811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:48.839237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] PATH_INFO: `/v3/users/f458111d5906467fadd22de38f32cf59` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:48.840515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f458111d5906467fadd22de38f32cf59)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:48.842515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:48.877215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:48.928772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f458111d5906467fadd22de38f32cf59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.929870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f458111d5906467fadd22de38f32cf59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.952864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f458111d5906467fadd22de38f32cf59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.965455 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f458111d5906467fadd22de38f32cf59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.966459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f458111d5906467fadd22de38f32cf59'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:48.991834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1144/2349] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:43:48 2022] DELETE /identity/v3/users/f458111d5906467fadd22de38f32cf59 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:49.395126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2e010-3de0-4161-b2a5-09e728564677 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:49.395718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2e010-3de0-4161-b2a5-09e728564677 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:49.395718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2e010-3de0-4161-b2a5-09e728564677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:49.424291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7aa2e010-3de0-4161-b2a5-09e728564677 None None] MFA Rules not processed for user `76c1fab92b854cd8937cc80818159191`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:49.425530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7aa2e010-3de0-4161-b2a5-09e728564677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:49.436330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:49.437889 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:49.518180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1206/2350] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:49.579364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c84b3d-867e-43f3-9f27-b8e3487d1f1e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:49.586705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:49.587386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:49.588363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:49.588654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] PATH_INFO: `/v3/projects/e9414c57cfca4e9b9b22472e24e78b2c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:49.590065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9414c57cfca4e9b9b22472e24e78b2c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:49.598410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:49.686623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9414c57cfca4e9b9b22472e24e78b2c', 'request_id': 'req-2800e2e2-0308-49c0-b390-fa08b05d9162'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:49.693057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9414c57cfca4e9b9b22472e24e78b2c', 'request_id': 'req-2800e2e2-0308-49c0-b390-fa08b05d9162'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:49.709513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9414c57cfca4e9b9b22472e24e78b2c', 'request_id': 'req-2800e2e2-0308-49c0-b390-fa08b05d9162'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:49.726313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] The token cache is being invalidate because project e9414c57cfca4e9b9b22472e24e78b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:43:49.726618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:49.727666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2800e2e2-0308-49c0-b390-fa08b05d9162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:43:49.740438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1145/2351] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:43:49 2022] DELETE /identity/v3/projects/e9414c57cfca4e9b9b22472e24e78b2c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:43:51.374353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:51.375269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:51.375798 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:51.396221 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] MFA Rules not processed for user `4b0aed0278344119aae6906f4f52b1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:51.397806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:51.434971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:51.465952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1207/2352] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:53.169047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:53.169047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:53.169047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:53.200741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] MFA Rules not processed for user `4294d6dff5054da7afcb37b8b28f1a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:43:53.202038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:53.231038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:43:53.316531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1146/2353] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:43:59.784319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:43:59.785852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:59.853967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0600ac-9ce4-4dba-9d8b-79cce5a08a8e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:43:59.857939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:43:59.858153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:43:59.858241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:43:59.858618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:43:59.860898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:43:59.957955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:43:59.959434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:43:59.971883 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1208/2354] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:43:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:00.839799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:00.856908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab19d0-3fb5-4e23-bd61-c420c09c4264 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:00.865801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:00.866584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:00.866960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:00.867342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:00.870438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:00.891653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:00.894053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69253c45-05ad-4645-a980-a1aaf467ad1c service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:00.908300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1147/2355] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:02.829177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:02.830769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:02.843389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf974cc-7594-413e-82f1-ed9893f25bfa service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:02.847362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:02.848088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:02.848361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:02.848642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:02.850454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:02.932143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:02.933752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:02.947380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1209/2356] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:44:10.901829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=18 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:44:12.942011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=93 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:44:33.120857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:33.121757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:33.121757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:33.142677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] MFA Rules not processed for user `1ac7d3acf1204a9f98f9ee14c2b46a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:44:33.144254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:33.207988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1148/2357] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:44:33.573556 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:33.581615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d3bdd-a2b7-479e-b1df-1750246e7705 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:33.585974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:33.586606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:33.586879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:33.586879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:33.589307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:33.600483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:33.601936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:33.611270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1210/2358] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:33.969653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:33.971293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:34.050043 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f2401-414e-4476-9f4f-542d8071169a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:34.055237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:34.055950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:34.056226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:34.056497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:34.058643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:34.071288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:34.073225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:34.084178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1149/2359] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:35.975169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:35.991995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db5f71-30f7-44fe-a6a5-bfcbe0f32128 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:35.996248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:35.996914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:35.997182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:35.997446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:35.999544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:36.010979 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:36.012655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:36.021670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1211/2360] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:37.415608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:37.417342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:37.434500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c70f22-3dc7-46e0-b1a3-d8b10f574b5f service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:37.440696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:37.441337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:37.441597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:37.441877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:37.444080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:37.456070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:37.457679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:37.468229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1150/2361] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:40.932311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:40.933977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:40.997837 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bedf9-6582-4341-95fd-60640586bc43 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:41.001851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:41.002472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:41.002796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:41.003007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:41.005007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:41.017281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:41.019040 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:41.030400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1212/2362] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:47.463075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=63 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:44:50.086457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:50.088929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:50.104898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d9a79-cbdd-4185-bab4-5bd0c943d6a7 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:50.110921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:50.111880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:50.112156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:50.112485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:50.114555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:50.127882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:50.129517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:50.141115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1151/2363] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:51.025349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=38 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:44:52.246157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:52.260232 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b04f4e-4bab-4fd5-bac2-106ab1e5cbd9 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:52.266749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:52.267406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:52.267679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:52.267998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:52.270103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:52.347214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:52.348897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:52.361818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1213/2364] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:55.256115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:55.268262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0957507-a1c3-4b4d-b193-cf351328f5f7 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:55.275940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:55.276512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:55.276791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:55.277060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:55.279200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:55.291309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:55.292943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:55.302687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1152/2365] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:58.961814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:58.977130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9f0a-f71b-4524-adbd-bcd083c311fd service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:58.984543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:58.985191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:58.985469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:58.985738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:58.988243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:59.003844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:59.005373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:59.017069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1214/2366] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:44:59.507742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:44:59.521184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427106-7b0d-41ff-92eb-757414e105e1 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:44:59.526450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:44:59.527114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:44:59.527417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:44:59.527684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:44:59.529636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:44:59.614108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:44:59.615571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:44:59.627329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1153/2367] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:45:03.439054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:03.441406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:03.467913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b437b-8284-4fdb-af18-11002c13388e service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:03.475161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:03.476730 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:03.477306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:03.477817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] PATH_INFO: `/v3/users/4294d6dff5054da7afcb37b8b28f1a01` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:03.479900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4294d6dff5054da7afcb37b8b28f1a01)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:03.483651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:03.565033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4294d6dff5054da7afcb37b8b28f1a01'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:03.565833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4294d6dff5054da7afcb37b8b28f1a01'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:03.577024 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4294d6dff5054da7afcb37b8b28f1a01'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:03.603590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4294d6dff5054da7afcb37b8b28f1a01'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:03.622210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-220f7592-f6ba-4422-a4d3-0a6f442cfeaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4294d6dff5054da7afcb37b8b28f1a01'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:03.632737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1215/2368] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:03 2022] DELETE /identity/v3/users/4294d6dff5054da7afcb37b8b28f1a01 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:03.865064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:03.878245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665133b7-dae7-4271-bfc9-10c33a0c148d service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:03.949006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:03.949673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:03.949951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:03.950220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] PATH_INFO: `/v3/projects/dbd6814591334db58a37092eb04729fb` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:03.951508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd6814591334db58a37092eb04729fb)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:03.953534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.010365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd6814591334db58a37092eb04729fb', 'request_id': 'req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:04.016128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd6814591334db58a37092eb04729fb', 'request_id': 'req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:04.026381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd6814591334db58a37092eb04729fb', 'request_id': 'req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:04.034901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:04.041675 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] The token cache is being invalidate because project dbd6814591334db58a37092eb04729fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:04.041946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:04.043075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f8ba83ef-2500-4201-9a5f-326b99cf3fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:04.045008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1154/2369] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:03 2022] DELETE /identity/v3/projects/dbd6814591334db58a37092eb04729fb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:04.050801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.051050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.051347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.072664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:04.073907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:04.127289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:04.138462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1216/2370] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:04.146162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.146597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.146712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.169319 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:04.170615 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:04.179044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1155/2371] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:04.184284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.194699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38f5c-b81b-4a40-9f6b-241057fa33b0 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.206395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.207053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.207334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.207603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.208353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:04.208665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.210443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.230497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1217/2372] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:04.231944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.242127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e228e-a44c-4eee-9223-0f466cdf7b34 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.247129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.247822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.248070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.248393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.249013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.250965 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.284505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1156/2373] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:04 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:04.289655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.300685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c6e2-0518-4241-a10e-f5b0bfadcc25 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.365872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.366496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.366862 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.367184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.367964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.369902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.397079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1218/2374] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:04 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:04.398398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.409864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dd8e5-b2b7-48b6-afc9-68864567fecb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.415433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.416095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.416386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.416689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.417402 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.419140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.430182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1157/2375] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:45:04.434693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.445740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff03384-9821-4037-b980-84f886d97907 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.450896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70317a5-6c56-4949-a483-1437b4d1d9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.451533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70317a5-6c56-4949-a483-1437b4d1d9be admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.451853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70317a5-6c56-4949-a483-1437b4d1d9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.452124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70317a5-6c56-4949-a483-1437b4d1d9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.452856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70317a5-6c56-4949-a483-1437b4d1d9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.454467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70317a5-6c56-4949-a483-1437b4d1d9be admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.467054 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1219/2376] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:04.471607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.482915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521949c-7b69-4741-8632-91c12724e2c9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.487934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.488622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.488894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.489164 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] PATH_INFO: `/v3/projects/c3601a4f15d84ea6b32e39f5e5a0f462/users/63a00b55295240d3a8b36a8cc156f667/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.513802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3601a4f15d84ea6b32e39f5e5a0f462, user_id=63a00b55295240d3a8b36a8cc156f667, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.516519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.533245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1158/2377] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:04 2022] PUT /identity/v3/projects/c3601a4f15d84ea6b32e39f5e5a0f462/users/63a00b55295240d3a8b36a8cc156f667/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:04.545279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.545542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.545821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.580100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] MFA Rules not processed for user `63a00b55295240d3a8b36a8cc156f667`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:04.581409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:04.670147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1220/2378] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:04.690546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:04.692985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:04.817706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1cd83-30be-47be-bd93-e31bec9b31bb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:04.824340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca79a5-fea5-4b5c-b7ef-f44c38cec938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:04.825037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca79a5-fea5-4b5c-b7ef-f44c38cec938 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:04.825315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca79a5-fea5-4b5c-b7ef-f44c38cec938 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:04.825584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca79a5-fea5-4b5c-b7ef-f44c38cec938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:04.834695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca79a5-fea5-4b5c-b7ef-f44c38cec938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:04.837018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca79a5-fea5-4b5c-b7ef-f44c38cec938 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:04.849040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1159/2379] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:45:12.372441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:12.428331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:12.441111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfea73-f4be-4c17-ba94-40e35385c9eb None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:12.446183 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:12.446850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:12.447154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:12.447424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] PATH_INFO: `/v3/users/63a00b55295240d3a8b36a8cc156f667` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:12.448631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a00b55295240d3a8b36a8cc156f667)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:12.450620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:12.502025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a00b55295240d3a8b36a8cc156f667'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:12.502717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a00b55295240d3a8b36a8cc156f667'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:12.512470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a00b55295240d3a8b36a8cc156f667'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:12.526376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a00b55295240d3a8b36a8cc156f667'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:12.534244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a00b55295240d3a8b36a8cc156f667'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:12.541472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1221/2380] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:12 2022] DELETE /identity/v3/users/63a00b55295240d3a8b36a8cc156f667 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:12.546132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:12.546701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:12.546701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:12.571604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:12.573086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:12.643611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1160/2381] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:12.656400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:12.658202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:12.727413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e7bee-11d4-4c53-9b15-e91bbe57b52d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:12.731255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff06a-bdde-43a2-8da3-9c9da48119ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:12.731926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff06a-bdde-43a2-8da3-9c9da48119ce service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:12.732204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff06a-bdde-43a2-8da3-9c9da48119ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:12.732476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff06a-bdde-43a2-8da3-9c9da48119ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:12.744350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfff06a-bdde-43a2-8da3-9c9da48119ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:12.745854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfff06a-bdde-43a2-8da3-9c9da48119ce service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:12.758135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1222/2382] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:13.083754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.085790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:13.104253 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56a24e-ecd5-48f3-815b-8a32de4955ee None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.114460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.115142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.115410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.115682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] PATH_INFO: `/v3/projects/c3601a4f15d84ea6b32e39f5e5a0f462` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.117485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3601a4f15d84ea6b32e39f5e5a0f462)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.119513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.144888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:13.185295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3601a4f15d84ea6b32e39f5e5a0f462', 'request_id': 'req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:13.200428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3601a4f15d84ea6b32e39f5e5a0f462', 'request_id': 'req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:13.211588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3601a4f15d84ea6b32e39f5e5a0f462', 'request_id': 'req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:13.234605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] The token cache is being invalidate because project c3601a4f15d84ea6b32e39f5e5a0f462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:13.234974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:13.235754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-db5fc6b0-762e-48e8-b72d-f8b6d19e79cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:13.237716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1161/2383] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:13 2022] DELETE /identity/v3/projects/c3601a4f15d84ea6b32e39f5e5a0f462 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:13.243907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.243907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.243907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.269309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:13.270664 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:13.353648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1223/2384] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:13.363477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.363741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.364063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.386697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:13.388029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:13.396602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1162/2385] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:13.401824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.413071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b9f70-c750-44a0-9286-0d7b8ce08a8e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.418336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.418987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.419217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.419484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.420256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:13.420550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.422216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.435914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1224/2386] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:13.440742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.451699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d726d-7779-4adb-a5f6-8da44e49816e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.457994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.458864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.459142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.459458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.460674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.463150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.503851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1163/2387] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:13 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:13.508627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.514320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:13.522488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ef3d-c892-482c-a0b6-5edf954c2cef admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.588412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.589455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.589711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.589927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.590636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.592602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.617897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1225/2388] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:13 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:13.623002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.634395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62ba37-303a-4c5a-849d-d3469bdfb925 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.639683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.640338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.640611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.640885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.641626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.643477 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.654862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1164/2389] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:13.659536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.671192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d22d8-3a1e-46af-a691-2df74c5e4ac9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.676154 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe0d5-481a-4639-9045-9c523c21fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.676807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe0d5-481a-4639-9045-9c523c21fb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.677080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe0d5-481a-4639-9045-9c523c21fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.677345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe0d5-481a-4639-9045-9c523c21fb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.678118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afe0d5-481a-4639-9045-9c523c21fb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.680095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afe0d5-481a-4639-9045-9c523c21fb07 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.691817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1226/2390] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:13.696380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.707600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160264c-2b02-4c9a-b6ef-a2d80eed7bb2 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.713366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.714514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.714858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.715120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] PATH_INFO: `/v3/projects/1aa0b8b7b2c348ca9f99f993b9d6759b/users/621d7e0cc8ee40bf9bf14b5c42180c69/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.732374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa0b8b7b2c348ca9f99f993b9d6759b, user_id=621d7e0cc8ee40bf9bf14b5c42180c69, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.735082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.751246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1165/2391] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:13 2022] PUT /identity/v3/projects/1aa0b8b7b2c348ca9f99f993b9d6759b/users/621d7e0cc8ee40bf9bf14b5c42180c69/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:13.762211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077637e-1db1-4e43-81ec-ccac7f066230 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.762454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077637e-1db1-4e43-81ec-ccac7f066230 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.762801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077637e-1db1-4e43-81ec-ccac7f066230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.799912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e077637e-1db1-4e43-81ec-ccac7f066230 None None] MFA Rules not processed for user `621d7e0cc8ee40bf9bf14b5c42180c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:13.801464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e077637e-1db1-4e43-81ec-ccac7f066230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:13.878745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1227/2392] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:13.892367 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:13.894370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:13.956853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65a7c-e3ad-432b-9146-029ba1e48472 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:13.960655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:13.961286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:13.961588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:13.961858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:13.972059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:13.973531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:13.985149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1166/2393] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:19.097488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:19.099016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:19.099676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:19.132362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:19.133722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:19.248225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1228/2394] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:45:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:19.338277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:19.341429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:19.419336 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:19.467821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a9cc4-9bd0-46ac-b2a1-6a35be05f11b service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:19.477500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279580e-ed21-4882-94d4-163436357908 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:19.478534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279580e-ed21-4882-94d4-163436357908 service swift] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:19.478975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279580e-ed21-4882-94d4-163436357908 service swift] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:19.479498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279580e-ed21-4882-94d4-163436357908 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:19.496709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279580e-ed21-4882-94d4-163436357908 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:19.499948 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279580e-ed21-4882-94d4-163436357908 service swift] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:19.518690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1167/2395] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:45:20.989503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:20.991950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:21.061871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:21.229878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396600f9-6168-4b96-b0aa-01c74b2c057e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:21.240519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:21.240848 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:21.241429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:21.241953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:21.246951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:21.271955 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:21.280756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3983340-9cf0-4ca2-8064-af849e070386 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:21.305030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1229/2396] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:23.842394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279580e-ed21-4882-94d4-163436357908 service swift] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:23.940163 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279580e-ed21-4882-94d4-163436357908 service swift] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:23.947653 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:23.948702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:23.950485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:23.952259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] PATH_INFO: `/v3/users/621d7e0cc8ee40bf9bf14b5c42180c69` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:23.955672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] RBAC: Authorizing `identity:delete_user(user_id=621d7e0cc8ee40bf9bf14b5c42180c69)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:23.960524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:24.064684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621d7e0cc8ee40bf9bf14b5c42180c69'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:24.065997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621d7e0cc8ee40bf9bf14b5c42180c69'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:24.102928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621d7e0cc8ee40bf9bf14b5c42180c69'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:24.116484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621d7e0cc8ee40bf9bf14b5c42180c69'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:24.117786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621d7e0cc8ee40bf9bf14b5c42180c69'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:24.153142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1168/2397] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:23 2022] DELETE /identity/v3/users/621d7e0cc8ee40bf9bf14b5c42180c69 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:24.162329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:24.165476 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:24.166241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:24.212045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:24.214326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:24.311792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1230/2398] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:24.323430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:24.326223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:24.449235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d9af4-37ee-4292-b37e-4d20c23e637a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:24.456505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1144a65-fd6e-4f58-829b-53454f06be29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:24.457359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144a65-fd6e-4f58-829b-53454f06be29 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:24.457652 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144a65-fd6e-4f58-829b-53454f06be29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:24.457908 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144a65-fd6e-4f58-829b-53454f06be29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:24.474961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:24.482849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144a65-fd6e-4f58-829b-53454f06be29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:24.485649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144a65-fd6e-4f58-829b-53454f06be29 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:24.499353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1169/2399] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:25.364962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:25.366544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:25.409276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f62f8-299f-42a1-a3d6-1f17ffeaa73c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:25.421576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:25.423096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:25.423692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:25.424380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] PATH_INFO: `/v3/projects/1aa0b8b7b2c348ca9f99f993b9d6759b` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:25.430305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa0b8b7b2c348ca9f99f993b9d6759b)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:25.435400 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:25.561809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa0b8b7b2c348ca9f99f993b9d6759b', 'request_id': 'req-41e8405c-4867-4c74-8d93-be31ef047436'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:25.573900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa0b8b7b2c348ca9f99f993b9d6759b', 'request_id': 'req-41e8405c-4867-4c74-8d93-be31ef047436'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:25.594772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa0b8b7b2c348ca9f99f993b9d6759b', 'request_id': 'req-41e8405c-4867-4c74-8d93-be31ef047436'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:25.632878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] The token cache is being invalidate because project 1aa0b8b7b2c348ca9f99f993b9d6759b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:25.633683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:25.637732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-41e8405c-4867-4c74-8d93-be31ef047436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:25.645564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1231/2400] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:25 2022] DELETE /identity/v3/projects/1aa0b8b7b2c348ca9f99f993b9d6759b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:25.650718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:25.651582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:25.651582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:25.695061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:25.697001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:25.850182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1170/2401] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:25.860001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:25.860383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:25.860772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:25.909776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:25.915429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:25.944079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1232/2402] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:25.953748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:25.977452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3a786-9dab-4080-a2a1-20d95ab36560 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:25.994502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:26.002904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.004039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.005585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.006749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:26.007188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:26.010329 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:26.049110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1171/2403] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:26.049434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:26.064029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911c9eb-40fe-49c2-ad10-a649dc730844 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:26.079429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:26.080370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.080755 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.081051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.081886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:26.083957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:26.104351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:26.135491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1233/2404] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:26 2022] POST /identity/v3/projects => generated 403 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:26.138621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:26.154587 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024414c-345e-4aaa-8166-732c23494afb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:26.319529 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:26.320444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.320610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.320803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.321428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:26.323561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:26.396608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:26.400008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1172/2405] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:26 2022] POST /identity/v3/users => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:26.422152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63ec8-fcdc-4bdd-9346-25f66c40fae4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:26.438553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:26.443507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.448044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.448999 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.451106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:26.453168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:26.481110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1234/2406] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:45:26.484563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:26.500323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113f458-4cdb-4220-8939-1038cf3df902 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:26.527803 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5af26-6add-489c-a4ce-587437f22073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:26.528464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5af26-6add-489c-a4ce-587437f22073 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.528744 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5af26-6add-489c-a4ce-587437f22073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.529016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5af26-6add-489c-a4ce-587437f22073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.529738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5af26-6add-489c-a4ce-587437f22073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:26.531378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5af26-6add-489c-a4ce-587437f22073 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:26.549343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1173/2407] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:26.553566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:26.582261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b69de-eb66-4507-b626-f3a3fedfd3fd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:26.603408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:26.607621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.608155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.608656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] PATH_INFO: `/v3/projects/aa1ca824c9e547458580a8be4dedad8d/users/5ca6c50a2319449ebdcbaf5fdfba8af1/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.640978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1ca824c9e547458580a8be4dedad8d, user_id=5ca6c50a2319449ebdcbaf5fdfba8af1, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:26.644443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:26.691827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1235/2408] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:26 2022] PUT /identity/v3/projects/aa1ca824c9e547458580a8be4dedad8d/users/5ca6c50a2319449ebdcbaf5fdfba8af1/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:26.695680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93411-e37a-44d3-99e6-fae016e40567 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:26.696661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93411-e37a-44d3-99e6-fae016e40567 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:26.697064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93411-e37a-44d3-99e6-fae016e40567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:26.721400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d9a93411-e37a-44d3-99e6-fae016e40567 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:26.722859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d9a93411-e37a-44d3-99e6-fae016e40567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:26.824983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:26.836690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1174/2409] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:26.850769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:26.853016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:27.080406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac6856-4d28-4e38-967f-38ef68baf209 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:27.090174 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:27.090941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:27.091331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:27.091890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:27.120975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:27.123207 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:27.144684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1236/2410] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:28.767631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:28.768248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:28.768807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:28.853538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] MFA Rules not processed for user `5ca6c50a2319449ebdcbaf5fdfba8af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:28.855799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:29.037412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1175/2411] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:29.045565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:29.059233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1372e83-a86a-4173-a158-34f7d20581a8 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:29.064825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:29.065469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:29.065747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:29.066029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:29.066706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:29.069456 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:29.112454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1237/2412] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:29 2022] POST /identity/v3/projects => generated 403 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:29.116435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:29.129910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e860d-4278-41ea-8690-51b06f31bef8 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:29.253016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:29.254963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:29.255754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:29.257383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:29.258777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:29.270609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:29.317136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1176/2413] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:29 2022] POST /identity/v3/users => generated 308 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:29.324375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:29.342562 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525070-922c-4315-a227-c60b8b3807bf admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:29.359524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:29.360251 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:29.360701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:29.361246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:29.362164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:29.364151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:29.391228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1238/2414] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:29.392807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:29.431755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29093ef3-930b-4fde-af3d-0f58870018c7 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:29.439555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:29.444452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:29.445351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:29.445663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] PATH_INFO: `/v3/projects/d6fe577e4d704831894f8e7c1ef27eb9/users/2a3b6066a13444f89e56993dc9dc47e4/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:29.469360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fe577e4d704831894f8e7c1ef27eb9, user_id=2a3b6066a13444f89e56993dc9dc47e4, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:29.481437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:29.515485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1177/2415] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:29 2022] PUT /identity/v3/projects/d6fe577e4d704831894f8e7c1ef27eb9/users/2a3b6066a13444f89e56993dc9dc47e4/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:29.519567 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:29.531593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25922ac1-bfa1-4af7-8eb9-37ec153f2d71 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:29.624440 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8e3c-9d30-4a0c-a437-95ddc61320dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:29.625206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8e3c-9d30-4a0c-a437-95ddc61320dc admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:29.625559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8e3c-9d30-4a0c-a437-95ddc61320dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:29.625933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8e3c-9d30-4a0c-a437-95ddc61320dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:29.626885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8e3c-9d30-4a0c-a437-95ddc61320dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:29.628666 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8e3c-9d30-4a0c-a437-95ddc61320dc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:29.642225 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1239/2416] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:29.647597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:29.664092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f326-cd18-44e0-ba83-481d50d491bc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:29.669373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:29.670064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:29.670339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:29.670609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] PATH_INFO: `/v3/projects/d6fe577e4d704831894f8e7c1ef27eb9/users/2a3b6066a13444f89e56993dc9dc47e4/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:29.674680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fe577e4d704831894f8e7c1ef27eb9, user_id=2a3b6066a13444f89e56993dc9dc47e4, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:29.683649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:29.714624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1178/2417] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:29 2022] PUT /identity/v3/projects/d6fe577e4d704831894f8e7c1ef27eb9/users/2a3b6066a13444f89e56993dc9dc47e4/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:30.860293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:30.861398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:30.861994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:30.894995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] MFA Rules not processed for user `2a3b6066a13444f89e56993dc9dc47e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:30.897731 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:30.919858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:31.083015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1240/2418] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:32.750172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:32.838906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:32.855813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0678cd-4078-455a-8386-21995be713e1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:32.866445 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:32.867980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:32.868640 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:32.869102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:32.872823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:32.891311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:32.974050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:32.974050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:32.992783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1179/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:32.993328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5bb43-2e54-4d62-85cf-8c73b4f4f24a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:32.999141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:33.000110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:33.000546 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:33.000989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] PATH_INFO: `/v3/users/5ca6c50a2319449ebdcbaf5fdfba8af1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:33.004252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca6c50a2319449ebdcbaf5fdfba8af1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:33.006787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:33.088728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca6c50a2319449ebdcbaf5fdfba8af1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:33.089770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca6c50a2319449ebdcbaf5fdfba8af1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:33.100681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca6c50a2319449ebdcbaf5fdfba8af1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:33.119027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca6c50a2319449ebdcbaf5fdfba8af1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:33.128958 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca6c50a2319449ebdcbaf5fdfba8af1'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:33.136494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1241/2420] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:32 2022] DELETE /identity/v3/users/5ca6c50a2319449ebdcbaf5fdfba8af1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:33.825051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:33.852064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab0f72-e91d-44f1-9f7a-7de0c6370b30 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:33.972049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:33.973470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:33.974106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:33.974865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] PATH_INFO: `/v3/projects/aa1ca824c9e547458580a8be4dedad8d` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:33.977338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1ca824c9e547458580a8be4dedad8d)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:33.981086 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:34.072182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1ca824c9e547458580a8be4dedad8d', 'request_id': 'req-d783fb78-20dc-4f32-a3fc-5418ac8a529e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.084169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1ca824c9e547458580a8be4dedad8d', 'request_id': 'req-d783fb78-20dc-4f32-a3fc-5418ac8a529e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.101742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1ca824c9e547458580a8be4dedad8d', 'request_id': 'req-d783fb78-20dc-4f32-a3fc-5418ac8a529e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.128810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] The token cache is being invalidate because project aa1ca824c9e547458580a8be4dedad8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:34.129911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.135226 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.139350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1180/2421] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:33 2022] DELETE /identity/v3/projects/aa1ca824c9e547458580a8be4dedad8d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:34.143239 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:34.145541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:34.159595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683effc-0dc4-44f6-9cc1-057cf0487f1b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:34.198526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:34.247482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:34.249323 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:34.250157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:34.250621 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] PATH_INFO: `/v3/users/2a3b6066a13444f89e56993dc9dc47e4` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:34.253242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a3b6066a13444f89e56993dc9dc47e4)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:34.258496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:34.348843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a3b6066a13444f89e56993dc9dc47e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.350107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a3b6066a13444f89e56993dc9dc47e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.368774 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a3b6066a13444f89e56993dc9dc47e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.399108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a3b6066a13444f89e56993dc9dc47e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.416821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-14b657cb-6038-4df1-ba90-7eb5eeb6694d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a3b6066a13444f89e56993dc9dc47e4'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:34.430688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1242/2422] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:34 2022] DELETE /identity/v3/users/2a3b6066a13444f89e56993dc9dc47e4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:34.952985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:34.957057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:34.980149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783fb78-20dc-4f32-a3fc-5418ac8a529e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.018743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:35.067003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.067690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.068909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.068909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] PATH_INFO: `/v3/projects/d6fe577e4d704831894f8e7c1ef27eb9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.069739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6fe577e4d704831894f8e7c1ef27eb9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.071798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.126798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6fe577e4d704831894f8e7c1ef27eb9', 'request_id': 'req-7bed966c-2eb2-473c-9d84-042b7c11b197'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:35.133066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6fe577e4d704831894f8e7c1ef27eb9', 'request_id': 'req-7bed966c-2eb2-473c-9d84-042b7c11b197'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:35.148787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6fe577e4d704831894f8e7c1ef27eb9', 'request_id': 'req-7bed966c-2eb2-473c-9d84-042b7c11b197'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:35.163939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] The token cache is being invalidate because project d6fe577e4d704831894f8e7c1ef27eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:35.164617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:35.166833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-7bed966c-2eb2-473c-9d84-042b7c11b197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:35.174542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1181/2423] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:34 2022] DELETE /identity/v3/projects/d6fe577e4d704831894f8e7c1ef27eb9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:45:35.176508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.177033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.177196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.203689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:35.206549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:35.307297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1243/2424] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:35.309912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.310573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.310857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.334789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:35.336320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:35.345170 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1182/2425] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:35.350429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:35.361807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821c88f-916d-4e81-a117-1f59e98fb4ea None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.367068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.368170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.368489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.368825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.369601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:35.369988 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.371709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.387905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1244/2426] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:35.394888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:35.406967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33185f-110d-4ec1-82cf-7b0fe1d85ae1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.414181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.414981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.415354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.415721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.416589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.418736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.478114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1183/2427] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:35 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:35.483957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:35.496143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f5952-8adc-4b8f-aca7-a07690fe6d36 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.556333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:35.600643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.601827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.602210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.602703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.603930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.607668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.655740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1245/2428] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:35 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:35.665366 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:35.693911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37817b47-04f9-4c39-b9cd-721aa7444991 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.705242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.706302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.707912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.708549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.709999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.714952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.739917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1184/2429] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:35.746134 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:35.763947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94813b-c4a0-4e54-8046-4ee4e044a837 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.769739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70975f-202c-4d5e-bf83-e091ae6859d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.770829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70975f-202c-4d5e-bf83-e091ae6859d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.770829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70975f-202c-4d5e-bf83-e091ae6859d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.771299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70975f-202c-4d5e-bf83-e091ae6859d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.772494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70975f-202c-4d5e-bf83-e091ae6859d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.774422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70975f-202c-4d5e-bf83-e091ae6859d3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.792447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1246/2430] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:35.794247 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:35.811499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a7a70-7287-4949-b992-1dd04173e5ab admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:35.820446 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:35.821521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.822087 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.822645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] PATH_INFO: `/v3/projects/b2dadcbe5b4a43f4b72b103682476a94/users/8aa4ce0580fa4b3ba8eeb59dbb06e912/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.852839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2dadcbe5b4a43f4b72b103682476a94, user_id=8aa4ce0580fa4b3ba8eeb59dbb06e912, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:35.858203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:35.880591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1185/2431] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:35 2022] PUT /identity/v3/projects/b2dadcbe5b4a43f4b72b103682476a94/users/8aa4ce0580fa4b3ba8eeb59dbb06e912/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:35.892746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:35.893404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:35.893911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:35.941462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] MFA Rules not processed for user `8aa4ce0580fa4b3ba8eeb59dbb06e912`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:35.942772 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:36.036820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1247/2432] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:36.041316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.138090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:36.157568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcf3d5-fe7e-4344-b2e6-54aa9133dc31 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:36.164971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:36.165629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.165904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.166169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.166853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:36.168848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:36.201341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1186/2433] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:36 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:36.205490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.217106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c7ae6-55bb-4906-826a-f3667d8d8f55 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:36.292055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:36.293888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.293888 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.294255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.295373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:36.298306 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:36.344655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1248/2434] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:36 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:36.346531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.372039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211e21-078b-4ead-ac9f-ecb14a8834ad admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:36.388812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:36.389688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.389960 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.390273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.391349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:36.393637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:36.431097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1187/2435] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:36.432555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.462794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07de72-c861-40d5-8382-4d928a476064 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:36.479066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:36.479066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.479066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.479066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] PATH_INFO: `/v3/projects/c22d9fb40fda4312900b14eda8bf1526/users/0e4848058c1145fbac9ce4222e7a1fcd/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.501168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22d9fb40fda4312900b14eda8bf1526, user_id=0e4848058c1145fbac9ce4222e7a1fcd, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:36.504312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:36.523582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1249/2436] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:36 2022] PUT /identity/v3/projects/c22d9fb40fda4312900b14eda8bf1526/users/0e4848058c1145fbac9ce4222e7a1fcd/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:36.529536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.542392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7f287-7ffb-470c-995e-786bc4b6d05c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:36.633238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca7933-3b53-463d-b1cf-d83ec32ace94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:36.634090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca7933-3b53-463d-b1cf-d83ec32ace94 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.635357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca7933-3b53-463d-b1cf-d83ec32ace94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.635748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca7933-3b53-463d-b1cf-d83ec32ace94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.637032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca7933-3b53-463d-b1cf-d83ec32ace94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:36.638854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca7933-3b53-463d-b1cf-d83ec32ace94 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:36.659540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1188/2437] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:36.661000 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.683058 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a52652-668f-43b5-93d4-bf51067b1fb2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:36.689984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:36.690920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.691218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.691667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] PATH_INFO: `/v3/projects/c22d9fb40fda4312900b14eda8bf1526/users/0e4848058c1145fbac9ce4222e7a1fcd/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.695105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22d9fb40fda4312900b14eda8bf1526, user_id=0e4848058c1145fbac9ce4222e7a1fcd, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:36.699380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:36.707252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:36.723962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1250/2438] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:36 2022] PUT /identity/v3/projects/c22d9fb40fda4312900b14eda8bf1526/users/0e4848058c1145fbac9ce4222e7a1fcd/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:36.732456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:36.732874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:36.733337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:36.785646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] MFA Rules not processed for user `0e4848058c1145fbac9ce4222e7a1fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:36.786864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:36.902072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1189/2439] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:36.916626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:36.919248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:37.006434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6dc4d-00e5-4679-8522-42ba4e64dd95 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:37.012806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:37.013462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:37.013735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:37.014005 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:37.121696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:37.123290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:37.147014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1251/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:37.572475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:37.575324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:37.639435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:37.677075 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126273b-fe06-463b-970e-fe4d666303ca None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:37.686631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:37.687540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:37.687847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:37.688141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] PATH_INFO: `/v3/users/4b0aed0278344119aae6906f4f52b1e5` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:37.691309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0aed0278344119aae6906f4f52b1e5)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:37.700905 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:37.782581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0aed0278344119aae6906f4f52b1e5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:37.783351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0aed0278344119aae6906f4f52b1e5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:37.806374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0aed0278344119aae6906f4f52b1e5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:37.820775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0aed0278344119aae6906f4f52b1e5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:37.822388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9fba9f48-4d77-4acb-a699-90a73a0fcbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0aed0278344119aae6906f4f52b1e5'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:37.850180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1190/2441] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:37 2022] DELETE /identity/v3/users/4b0aed0278344119aae6906f4f52b1e5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:45:38.161992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:38.180975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92959298-da90-4ad0-a00d-0bafd109da8e service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:38.335642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:38.336619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:38.337029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:38.338749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] PATH_INFO: `/v3/projects/1b66d46c3e7d42b88f2dbdc01232fb03` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:38.356057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b66d46c3e7d42b88f2dbdc01232fb03)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:38.361395 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:38.437226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b66d46c3e7d42b88f2dbdc01232fb03', 'request_id': 'req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:38.441672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:38.448128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b66d46c3e7d42b88f2dbdc01232fb03', 'request_id': 'req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:38.461703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b66d46c3e7d42b88f2dbdc01232fb03', 'request_id': 'req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:38.483807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] The token cache is being invalidate because project 1b66d46c3e7d42b88f2dbdc01232fb03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:38.484106 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:38.485037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-51c4d3ac-ab3f-4981-a139-4ba7ffa9d375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:38.486928 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1252/2442] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:38 2022] DELETE /identity/v3/projects/1b66d46c3e7d42b88f2dbdc01232fb03 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:38.492617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:38.493269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:38.493589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:38.520437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:38.521304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:38.624589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1191/2443] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:38.633330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:38.633330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:38.633649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:38.690587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:38.692346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:38.704008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1253/2444] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:38.709116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:38.735025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfe7d7-8a31-4d91-8609-a9e359b1a428 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:38.747784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:38.749988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:38.750354 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:38.750845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:38.752627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:38.753635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:38.755607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:38.791522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1192/2445] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:38.793470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:38.815104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4946-bdc8-450f-966b-97b8a7a235fa None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:38.822328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:38.823480 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:38.823865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:38.824287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:38.825061 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:38.827171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:38.870475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1254/2446] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:38 2022] POST /identity/v3/projects => generated 403 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:38.879282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:38.898819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28a14-1ec1-4881-8eb6-ec7b2463344f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.004906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:39.021797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.022939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.023583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.024360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.025647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.029686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.076268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1193/2447] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:38 2022] POST /identity/v3/users => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:39.081991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.103441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180990-e2d8-475e-a652-a131f5236073 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.111003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.112028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.112506 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.112962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.114197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.117105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.157839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1255/2448] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:39.158992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.183959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e5b74-2a40-4f89-87f6-a2e4f7f0ee81 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.189302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9d5ce-d092-4f29-919f-958353daee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.190084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d5ce-d092-4f29-919f-958353daee6d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.190498 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d5ce-d092-4f29-919f-958353daee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.190828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9d5ce-d092-4f29-919f-958353daee6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.192344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d5ce-d092-4f29-919f-958353daee6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.194200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9d5ce-d092-4f29-919f-958353daee6d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.207322 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1194/2449] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:39.212431 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.226421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2565a5-2fda-4ddd-a7ad-cf71694d0802 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.232638 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.233474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.233833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.234157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] PATH_INFO: `/v3/projects/2e733e37ec2b4d99a5894ea4e20b5e92/users/6721594544c74e709605d95935e42fbd/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.268409 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e733e37ec2b4d99a5894ea4e20b5e92, user_id=6721594544c74e709605d95935e42fbd, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.273434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.296107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1256/2450] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:39 2022] PUT /identity/v3/projects/2e733e37ec2b4d99a5894ea4e20b5e92/users/6721594544c74e709605d95935e42fbd/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:39.315448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.315987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.316468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.389282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] MFA Rules not processed for user `6721594544c74e709605d95935e42fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:39.392079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:39.533028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1195/2451] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:39.538873 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.655645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa63b-d40d-47c8-950a-4e6f31b18cc3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.662048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.662835 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.663117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.663548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.664295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.666286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.722993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1257/2452] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:39 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:39.731290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.743976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e68ad-e675-40c7-8f61-fbc085599efc None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.822458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.823342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.823485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.823655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.824444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.826496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.858179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1196/2453] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:39 2022] POST /identity/v3/users => generated 309 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:39.863747 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.879390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1012c4-6620-4f37-904f-737da868b6de admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.886197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.887250 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.887856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.888396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.889338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.891368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.910825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1258/2454] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:45:39.914985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.926060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162fcda-65fd-4076-9c64-6d404ac7e1e1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:39.931506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:39.932347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:39.932521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:39.932778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] PATH_INFO: `/v3/projects/9e49ef92c9ef4e498d6a5ca3e9d2c01e/users/b562932a00a64268b6e257bc829fe2bb/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:39.953221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e49ef92c9ef4e498d6a5ca3e9d2c01e, user_id=b562932a00a64268b6e257bc829fe2bb, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:39.956066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:39.976369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1197/2455] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:39 2022] PUT /identity/v3/projects/9e49ef92c9ef4e498d6a5ca3e9d2c01e/users/b562932a00a64268b6e257bc829fe2bb/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:39.981375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:39.993019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c4f86-a803-4238-9e06-8031ff0f74f9 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:40.045363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:40.057469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091631e5-fe2f-4c0a-987e-343fb5ea0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:40.058466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091631e5-fe2f-4c0a-987e-343fb5ea0947 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:40.059003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091631e5-fe2f-4c0a-987e-343fb5ea0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:40.059555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091631e5-fe2f-4c0a-987e-343fb5ea0947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:40.060797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091631e5-fe2f-4c0a-987e-343fb5ea0947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:40.063153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091631e5-fe2f-4c0a-987e-343fb5ea0947 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:40.076280 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1259/2456] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:40.081609 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:40.095117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fa43a-199c-4622-8eb5-c31364bb77cf admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:40.100531 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:40.101241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:40.101515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:40.101783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] PATH_INFO: `/v3/projects/9e49ef92c9ef4e498d6a5ca3e9d2c01e/users/b562932a00a64268b6e257bc829fe2bb/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:40.103887 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e49ef92c9ef4e498d6a5ca3e9d2c01e, user_id=b562932a00a64268b6e257bc829fe2bb, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:40.106784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:40.118636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1198/2457] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:40 2022] PUT /identity/v3/projects/9e49ef92c9ef4e498d6a5ca3e9d2c01e/users/b562932a00a64268b6e257bc829fe2bb/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:40.131009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:40.131383 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:40.131752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:40.166991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] MFA Rules not processed for user `b562932a00a64268b6e257bc829fe2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:40.168177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:40.265404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1260/2458] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:40.278195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:40.280502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:40.299307 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:40.351100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bcae0-bd74-4a52-8db3-73b397f27cb1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:40.354955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce204971-585b-4836-8e63-54d77015017a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:40.355585 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204971-585b-4836-8e63-54d77015017a service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:40.355886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204971-585b-4836-8e63-54d77015017a service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:40.356154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204971-585b-4836-8e63-54d77015017a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:40.366320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204971-585b-4836-8e63-54d77015017a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:40.367839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204971-585b-4836-8e63-54d77015017a service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:40.377964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1199/2459] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:41.892117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:41.969654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fc838-72cd-49f8-9a1a-c8f8d495cf5c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:41.975382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:41.976130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:41.976417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:41.976675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] PATH_INFO: `/v3/users/6721594544c74e709605d95935e42fbd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:41.977962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6721594544c74e709605d95935e42fbd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:41.980014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:42.035261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6721594544c74e709605d95935e42fbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.035971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6721594544c74e709605d95935e42fbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.045474 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6721594544c74e709605d95935e42fbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.060173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6721594544c74e709605d95935e42fbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.065886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:42.069549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6721594544c74e709605d95935e42fbd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.079184 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1261/2460] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:41 2022] DELETE /identity/v3/users/6721594544c74e709605d95935e42fbd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:42.086254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:42.086830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:42.087130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:42.113534 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:42.115023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:42.184426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1200/2461] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:42.195262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:42.197294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:42.292222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75ee6c-8063-4bbe-892a-2498bf55274d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:42.292222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:42.292222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:42.292222 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:42.292879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:42.306440 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:42.309124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:42.309475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:42.312316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:42.331952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1262/2462] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:45:42.432748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b72247-9282-41ea-814b-5a14f3294b65 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:42.438583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:42.439535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:42.439991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:42.440424 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:42.442682 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:42.563348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:42.566113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:42.582141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1201/2463] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:42.709766 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:42.711478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:42.743408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719692a3-520a-47df-9f7f-3474af3479ed service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:42.750744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:42.751868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:42.752346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:42.752811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] PATH_INFO: `/v3/projects/2e733e37ec2b4d99a5894ea4e20b5e92` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:42.754781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e733e37ec2b4d99a5894ea4e20b5e92)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:42.758509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:42.827531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e733e37ec2b4d99a5894ea4e20b5e92', 'request_id': 'req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.834681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e733e37ec2b4d99a5894ea4e20b5e92', 'request_id': 'req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.844343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e733e37ec2b4d99a5894ea4e20b5e92', 'request_id': 'req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.857796 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] The token cache is being invalidate because project 2e733e37ec2b4d99a5894ea4e20b5e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:42.858139 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.858765 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:42.860467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1263/2464] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:42 2022] DELETE /identity/v3/projects/2e733e37ec2b4d99a5894ea4e20b5e92 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:42.864475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:42.876858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee2524-75e5-49b9-88e1-e7f69ed06d98 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:42.877796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:42.949458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:42.950094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:42.950375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:42.950648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] PATH_INFO: `/v3/users/b562932a00a64268b6e257bc829fe2bb` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:42.951930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b562932a00a64268b6e257bc829fe2bb)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:42.953869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:43.052781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b562932a00a64268b6e257bc829fe2bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.054115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b562932a00a64268b6e257bc829fe2bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.084089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b562932a00a64268b6e257bc829fe2bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.097130 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b562932a00a64268b6e257bc829fe2bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.097971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-370ff107-7d0a-4544-8160-29db83c157e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b562932a00a64268b6e257bc829fe2bb'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.116158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1202/2465] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:42 2022] DELETE /identity/v3/users/b562932a00a64268b6e257bc829fe2bb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:43.456432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:43.457998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:43.474561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa19159-ccb3-4fbb-812b-3e2b27a36a2d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:43.547071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:43.547733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:43.548044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:43.548312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] PATH_INFO: `/v3/projects/9e49ef92c9ef4e498d6a5ca3e9d2c01e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:43.549534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e49ef92c9ef4e498d6a5ca3e9d2c01e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:43.551445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:43.577667 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:43.608857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e49ef92c9ef4e498d6a5ca3e9d2c01e', 'request_id': 'req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.616270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e49ef92c9ef4e498d6a5ca3e9d2c01e', 'request_id': 'req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.625851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e49ef92c9ef4e498d6a5ca3e9d2c01e', 'request_id': 'req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.638229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] The token cache is being invalidate because project 9e49ef92c9ef4e498d6a5ca3e9d2c01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:43.638515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.639210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-3f3d3f48-7130-4a4b-9b33-6d24e3e51a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:43.640806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1264/2466] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:43 2022] DELETE /identity/v3/projects/9e49ef92c9ef4e498d6a5ca3e9d2c01e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:43.646267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:43.646658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:43.646966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:43.671230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:43.672626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:43.740969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1203/2467] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:43.748932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:43.749194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:43.749479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:43.772003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:43.772842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:43.782509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1265/2468] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:43.787140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:43.797386 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c74c0-3a5a-45e8-a387-03658442df42 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:43.802221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:43.802995 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:43.803483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:43.803806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:43.804519 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:43.804817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:43.806451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:43.819049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1204/2469] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:43.824070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:43.833902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bdf7-24d8-43fd-bcb9-4fa4e1b7b30e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:43.839203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:43.839874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:43.840142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:43.840426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:43.841111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:43.843044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:43.878503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1266/2470] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:43 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:43.883444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:43.894800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef2544-db40-4f95-9f67-978e57fb89f3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:43.905385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:43.965838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:43.966504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:43.966791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:43.967078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:43.967797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:43.969785 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:43.997487 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1205/2471] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:43 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:44.002122 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.013933 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0c68-b993-481d-b44a-503f4b3d1ee8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.019384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.021483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.021483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.021483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.021483 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.023473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.037398 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1267/2472] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:44.041527 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.054184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118a037-6ca0-44f9-891d-a9757413533a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.059918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60461b26-92a5-4787-b479-2acf5f46b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.060571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461b26-92a5-4787-b479-2acf5f46b586 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.060847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461b26-92a5-4787-b479-2acf5f46b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.061117 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60461b26-92a5-4787-b479-2acf5f46b586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.061835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60461b26-92a5-4787-b479-2acf5f46b586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.063510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60461b26-92a5-4787-b479-2acf5f46b586 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.077254 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1206/2473] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:44.081696 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.093481 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90835f-362d-4ba4-a5b8-be977f79d0f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.098145 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.098845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.099120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.099484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] PATH_INFO: `/v3/projects/34cb016a3ca1495a8edc26d75ee81da7/users/dea6bb5d583a412e8155c33640a10f8c/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.121072 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cb016a3ca1495a8edc26d75ee81da7, user_id=dea6bb5d583a412e8155c33640a10f8c, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.123748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.140936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1268/2474] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:44 2022] PUT /identity/v3/projects/34cb016a3ca1495a8edc26d75ee81da7/users/dea6bb5d583a412e8155c33640a10f8c/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:44.154650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.155298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.155298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.186721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] MFA Rules not processed for user `dea6bb5d583a412e8155c33640a10f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:44.188079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:44.272375 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1207/2475] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:44.283416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.357746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b915c36-b86c-4cb2-a24c-3a6049af4fb5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.362833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.363467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.363748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.364387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.365206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.368642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.406493 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1269/2476] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:44.410910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.423044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c11b01-56d6-405a-aedb-9828cab3c568 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.491790 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.492433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.492795 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.493174 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.493832 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.495746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.522988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1208/2477] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:44 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:44.524952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.537228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29b23b-051c-4e71-81ff-e3037a5e075b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.543152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.543833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.544131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.544387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.545104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.546734 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.559173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1270/2478] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:45:44.563703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.574912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8b888-d7d7-4a51-8ef6-83d69105d059 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.580018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22425ff3-4aae-43ad-8590-e108de5c9e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.580754 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22425ff3-4aae-43ad-8590-e108de5c9e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.581147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22425ff3-4aae-43ad-8590-e108de5c9e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.581458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22425ff3-4aae-43ad-8590-e108de5c9e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.582283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22425ff3-4aae-43ad-8590-e108de5c9e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.584218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22425ff3-4aae-43ad-8590-e108de5c9e8d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.586997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:44.601191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1209/2479] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:44.602500 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.613512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047eed8-e786-466a-aa95-9dd61bd08f80 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.618529 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.619210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.619489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.619788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] PATH_INFO: `/v3/projects/a989b15d376c468b85e90c9d6ff8fe41/users/318bdafc698a443dbe1b22cba9e08590/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.637157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a989b15d376c468b85e90c9d6ff8fe41, user_id=318bdafc698a443dbe1b22cba9e08590, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.640151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.643690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:44.661116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1271/2480] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:44 2022] PUT /identity/v3/projects/a989b15d376c468b85e90c9d6ff8fe41/users/318bdafc698a443dbe1b22cba9e08590/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:45:44.672718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.673201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.673823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.708115 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] MFA Rules not processed for user `318bdafc698a443dbe1b22cba9e08590`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:44.709338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:44.790273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1210/2481] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:45:44.796169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.876357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840d10-cb98-4814-9b56-f643751890f1 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.881539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.882141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.882413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.882679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.883360 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:44.885517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:44.916697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1272/2482] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:45:44.921039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:44.932807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38981c-fc0c-4c89-a9a1-5be875570d1d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:44.996933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:44.997626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:44.997929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:44.998377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:44.999242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.001225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.029018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1211/2483] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:44 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:45.030384 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:45.042147 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312470da-09e0-43a7-bf05-6983816025a3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:45.053129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:45.054045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.054359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.054647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.055410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.057078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.071152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1273/2484] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:45.075781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:45.086868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d3aea-5ca3-4e86-bf02-37f67194e21f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:45.092135 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:45.092822 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.093102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.093379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] PATH_INFO: `/v3/projects/82c3746929074eb38d020867a5459958/users/5fd6e887d81040cd8835fbdaaf5c39d2/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.114285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c3746929074eb38d020867a5459958, user_id=5fd6e887d81040cd8835fbdaaf5c39d2, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.117458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.133709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1212/2485] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:45 2022] PUT /identity/v3/projects/82c3746929074eb38d020867a5459958/users/5fd6e887d81040cd8835fbdaaf5c39d2/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:45.140356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:45.152014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f093b-8ef3-4dc7-9556-f72a02ddde66 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:45.219751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e7050-fc3a-4200-be68-5a46f2da1ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:45.220548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e7050-fc3a-4200-be68-5a46f2da1ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.220920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e7050-fc3a-4200-be68-5a46f2da1ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.221335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e7050-fc3a-4200-be68-5a46f2da1ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.222109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e7050-fc3a-4200-be68-5a46f2da1ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.224008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e7050-fc3a-4200-be68-5a46f2da1ce4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.235277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1274/2486] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:45:45.239496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:45.251470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7e98c-4387-4925-800f-c1a1a855846a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:45.257688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:45.258382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.258656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.258922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] PATH_INFO: `/v3/projects/82c3746929074eb38d020867a5459958/users/5fd6e887d81040cd8835fbdaaf5c39d2/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.261388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c3746929074eb38d020867a5459958, user_id=5fd6e887d81040cd8835fbdaaf5c39d2, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.264201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.276564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1213/2487] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:45 2022] PUT /identity/v3/projects/82c3746929074eb38d020867a5459958/users/5fd6e887d81040cd8835fbdaaf5c39d2/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:45.287952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.288520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.288879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.319624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] MFA Rules not processed for user `5fd6e887d81040cd8835fbdaaf5c39d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:45.320813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:45.373369 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:45.394899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1275/2488] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:45.406770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:45.408996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:45.458870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:45.480648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5adaee-e915-470c-a159-69a4f2aab5b0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:45.485478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e37ff-61b4-4034-96d4-1ea6d77ac81e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:45.486280 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e37ff-61b4-4034-96d4-1ea6d77ac81e service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.486484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e37ff-61b4-4034-96d4-1ea6d77ac81e service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.486751 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e37ff-61b4-4034-96d4-1ea6d77ac81e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.502230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e37ff-61b4-4034-96d4-1ea6d77ac81e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.503789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e37ff-61b4-4034-96d4-1ea6d77ac81e service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.524940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1214/2489] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:45.577254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:45.579203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:45.592433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208de0d8-3e7b-4b1a-96db-cb261740685b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:45.597240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:45.597697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:45.598155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:45.598277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:45.663342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:45.664843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:45.674806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1276/2490] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:47.191563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:47.192400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:47.193722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:47.261756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:47.263239 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:47.391831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1215/2491] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Aug 1 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:47.405053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:47.409370 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:47.588166 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb5ac-2fdf-4268-b8fd-e00fc6884add service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:47.598967 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae731e-bef9-405d-abb1-59cce7daa002 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:47.600649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae731e-bef9-405d-abb1-59cce7daa002 service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:47.600649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae731e-bef9-405d-abb1-59cce7daa002 service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:47.602120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae731e-bef9-405d-abb1-59cce7daa002 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:47.621408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae731e-bef9-405d-abb1-59cce7daa002 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:47.624840 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae731e-bef9-405d-abb1-59cce7daa002 service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:47.637227 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1277/2492] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:45:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:45:49.380999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:49.512256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b29a5-316a-4a95-ae2b-0ecb475e4b21 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:49.527648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:49.528890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:49.529848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:49.533019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] PATH_INFO: `/v3/users/dea6bb5d583a412e8155c33640a10f8c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:49.535244 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea6bb5d583a412e8155c33640a10f8c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:49.538380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:49.637283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea6bb5d583a412e8155c33640a10f8c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:49.638076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea6bb5d583a412e8155c33640a10f8c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:49.684758 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea6bb5d583a412e8155c33640a10f8c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:49.712769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea6bb5d583a412e8155c33640a10f8c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:49.716649 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea6bb5d583a412e8155c33640a10f8c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:49.749920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1216/2493] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:49 2022] DELETE /identity/v3/users/dea6bb5d583a412e8155c33640a10f8c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:49.751522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:49.752471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:49.752471 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:49.779558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:49.780778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:49.957402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1278/2494] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:49.972248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:49.974651 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:49.991112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:50.119884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccf3d4-f2a6-48b6-9394-bd5e9072e8ea admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:50.130310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:50.134100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:50.136388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:50.139077 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:50.184892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:50.191412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:50.222969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1217/2495] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:50.895265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:50.897940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:50.922808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5dbc5-e5ec-47ab-a723-bac350e53394 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:50.932368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:50.933516 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:50.934023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:50.934353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] PATH_INFO: `/v3/projects/34cb016a3ca1495a8edc26d75ee81da7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:50.936389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34cb016a3ca1495a8edc26d75ee81da7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:50.940211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:50.973404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:51.039330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34cb016a3ca1495a8edc26d75ee81da7', 'request_id': 'req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.047110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34cb016a3ca1495a8edc26d75ee81da7', 'request_id': 'req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.058557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34cb016a3ca1495a8edc26d75ee81da7', 'request_id': 'req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.075515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] The token cache is being invalidate because project 34cb016a3ca1495a8edc26d75ee81da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:51.076093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.076683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.078975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1279/2496] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:50 2022] DELETE /identity/v3/projects/34cb016a3ca1495a8edc26d75ee81da7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:51.083633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:51.105558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc6e0b-3be3-46ba-9eb5-e591e77ff04d service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:51.311798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:51.317493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:51.320123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:51.320123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] PATH_INFO: `/v3/users/318bdafc698a443dbe1b22cba9e08590` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:51.324605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=318bdafc698a443dbe1b22cba9e08590)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:51.326835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:51.426321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318bdafc698a443dbe1b22cba9e08590'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.427217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318bdafc698a443dbe1b22cba9e08590'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.446756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318bdafc698a443dbe1b22cba9e08590'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.461743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318bdafc698a443dbe1b22cba9e08590'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.464208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318bdafc698a443dbe1b22cba9e08590'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:51.492577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1218/2497] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:51 2022] DELETE /identity/v3/users/318bdafc698a443dbe1b22cba9e08590 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:52.024778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:52.026441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:52.072132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c72f0-49a3-42a2-8432-cf2a07d69aa3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:52.160472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:52.161574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:52.162109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:52.162605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] PATH_INFO: `/v3/projects/a989b15d376c468b85e90c9d6ff8fe41` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:52.164533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a989b15d376c468b85e90c9d6ff8fe41)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:52.166892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:52.248626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a989b15d376c468b85e90c9d6ff8fe41', 'request_id': 'req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.262884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a989b15d376c468b85e90c9d6ff8fe41', 'request_id': 'req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.274677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a989b15d376c468b85e90c9d6ff8fe41', 'request_id': 'req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.288957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] The token cache is being invalidate because project a989b15d376c468b85e90c9d6ff8fe41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:52.289344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.290053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.292543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1280/2498] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:52 2022] DELETE /identity/v3/projects/a989b15d376c468b85e90c9d6ff8fe41 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:52.299042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:52.301537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:52.318378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fa453-97aa-4345-849f-c74bbca87f2e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:52.383716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:52.508655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:52.512081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:52.513929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:52.525521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] PATH_INFO: `/v3/users/5fd6e887d81040cd8835fbdaaf5c39d2` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:52.528819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd6e887d81040cd8835fbdaaf5c39d2)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:52.533210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:52.684716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd6e887d81040cd8835fbdaaf5c39d2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.686856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd6e887d81040cd8835fbdaaf5c39d2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.727272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd6e887d81040cd8835fbdaaf5c39d2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.748157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd6e887d81040cd8835fbdaaf5c39d2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.750556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a660c596-2f64-4797-aaf7-1a0b113bf215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd6e887d81040cd8835fbdaaf5c39d2'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:52.823695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1219/2499] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:52 2022] DELETE /identity/v3/users/5fd6e887d81040cd8835fbdaaf5c39d2 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:53.371237 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:53.374818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:53.399530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e085-44d7-4be3-a4bc-e4950c9fd3ca admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:53.477223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:53.550420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:53.551342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:53.552240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:53.552785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] PATH_INFO: `/v3/projects/82c3746929074eb38d020867a5459958` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:53.554725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c3746929074eb38d020867a5459958)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:53.557292 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:53.626803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c3746929074eb38d020867a5459958', 'request_id': 'req-874e84c7-dd45-480c-a06c-6245e5fb3ab9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:53.635534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c3746929074eb38d020867a5459958', 'request_id': 'req-874e84c7-dd45-480c-a06c-6245e5fb3ab9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:53.650962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c3746929074eb38d020867a5459958', 'request_id': 'req-874e84c7-dd45-480c-a06c-6245e5fb3ab9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:53.664633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] The token cache is being invalidate because project 82c3746929074eb38d020867a5459958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:45:53.664922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:53.665590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-874e84c7-dd45-480c-a06c-6245e5fb3ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:53.667346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1281/2500] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:53 2022] DELETE /identity/v3/projects/82c3746929074eb38d020867a5459958 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:53.680225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:53.681221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:53.681730 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:53.711338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:53.715964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:53.823631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1220/2501] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:53.832830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:53.833151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:53.834081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:53.889286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:53.892182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:53.912022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1282/2502] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:53.916645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:53.938661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378c2d7-fce2-4b59-bc82-4e4e6f209b04 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:53.946368 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:53.947526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:53.948233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:53.948370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:53.949326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:45:53.949842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:53.952352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.006233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1221/2503] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:45:54.006866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.021605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a3bc4-db34-4b54-8fc0-e8beaaf94d2b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:54.036992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:54.038151 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.038538 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.038907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.039738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:54.044468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.092839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1283/2504] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:45:54 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:54.097963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.111837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe5fb-d617-48ee-9d76-d55167597739 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:54.156639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:54.204711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:54.206059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.206481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.207051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.208084 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:54.211499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.246998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1222/2505] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:45:54 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:45:54.251960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.270824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfcd6d-7037-4c72-9c15-6649fe160519 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:54.277762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:54.279045 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.279717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.280397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.281576 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:54.284281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.304709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1284/2506] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:54.309062 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.323199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526c373-d5c9-4cfa-bcb6-a066079fb457 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:54.330406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae74b1-47e6-4c46-8f71-3411401ba730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:54.331929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae74b1-47e6-4c46-8f71-3411401ba730 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.332879 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae74b1-47e6-4c46-8f71-3411401ba730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.333817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae74b1-47e6-4c46-8f71-3411401ba730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.335718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae74b1-47e6-4c46-8f71-3411401ba730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:54.338697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae74b1-47e6-4c46-8f71-3411401ba730 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.356145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1223/2507] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:45:54.361435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.413583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659644d3-3405-49b9-93f9-0c17309f206e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:54.431574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:54.434623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.435371 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.435921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] PATH_INFO: `/v3/projects/e012f877d0da438ba9e2433436cc691a/users/888d9c44e51c428aab0ae7ac78d307b0/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.466302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e012f877d0da438ba9e2433436cc691a, user_id=888d9c44e51c428aab0ae7ac78d307b0, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:54.479013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.502725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1285/2508] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:45:54 2022] PUT /identity/v3/projects/e012f877d0da438ba9e2433436cc691a/users/888d9c44e51c428aab0ae7ac78d307b0/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:54.528878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.529224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.529568 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.608553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] MFA Rules not processed for user `888d9c44e51c428aab0ae7ac78d307b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:45:54.612214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:54.753894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1224/2509] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:45:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:45:54.785069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.788269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:54.924336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f27b3-7be0-4f36-942c-cca72b6a738a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:54.928853 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:54.929597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:54.929921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:54.930338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:54.932192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:54.935702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:54.942033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:54.944050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:54.956264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1286/2510] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:55.093118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b1ba8-ff2c-488b-8ae1-1ceb8f825850 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:55.101882 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:55.102557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:55.102836 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:55.103105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:55.105486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:55.213157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:55.226504 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:55.231284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:55.244939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1225/2511] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:59.434838 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:59.436769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:59.444130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:45:59.529387 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:59.531694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:59.545834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae54f62-3563-4626-bfe3-c4f895bc881c service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:59.551276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:59.552050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:59.552330 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:59.552603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] PATH_INFO: `/v3/users/1ac7d3acf1204a9f98f9ee14c2b46a5e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:59.553954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac7d3acf1204a9f98f9ee14c2b46a5e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:59.556194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:59.563448 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99699e8f-3a0e-48b9-98ed-31cdb964fc7d service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:45:59.572716 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:45:59.573705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:45:59.574095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:45:59.574478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:45:59.576884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:45:59.593437 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:45:59.595277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:45:59.612380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1287/2512] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:45:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:45:59.623053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac7d3acf1204a9f98f9ee14c2b46a5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:59.623843 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac7d3acf1204a9f98f9ee14c2b46a5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:59.645600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac7d3acf1204a9f98f9ee14c2b46a5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:59.660013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac7d3acf1204a9f98f9ee14c2b46a5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:59.661314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2bb8d598-9259-490f-aa92-3671e24c2c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac7d3acf1204a9f98f9ee14c2b46a5e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:45:59.691040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1226/2513] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:45:59 2022] DELETE /identity/v3/users/1ac7d3acf1204a9f98f9ee14c2b46a5e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:45:59.919916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:45:59.938947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e5c1-dcad-48d4-ac9d-3202040a5cc5 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.026062 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.027023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.027443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.027898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] PATH_INFO: `/v3/projects/2f46e472f8614c5e847ceb225cc9bff1` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.029699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f46e472f8614c5e847ceb225cc9bff1)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.031686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.095418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f46e472f8614c5e847ceb225cc9bff1', 'request_id': 'req-a12e797e-64ad-4c6b-8c94-b33399847ae9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:00.106678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f46e472f8614c5e847ceb225cc9bff1', 'request_id': 'req-a12e797e-64ad-4c6b-8c94-b33399847ae9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:00.118238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f46e472f8614c5e847ceb225cc9bff1', 'request_id': 'req-a12e797e-64ad-4c6b-8c94-b33399847ae9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:00.131069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] The token cache is being invalidate because project 2f46e472f8614c5e847ceb225cc9bff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:00.131406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:00.132168 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a12e797e-64ad-4c6b-8c94-b33399847ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:00.133827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1288/2514] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:45:59 2022] DELETE /identity/v3/projects/2f46e472f8614c5e847ceb225cc9bff1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:00.139242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.139528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.139853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.165232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:00.166391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:00.254678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1227/2515] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:00.264525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.264815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.265087 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.287293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:00.288590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:00.297496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1289/2516] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:00.302384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.313842 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e43bb1-1ae3-4951-ac73-65632034b1ed None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.319720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.320563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.320707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.320918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.321618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:00.321919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.323551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.336841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1228/2517] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:00.341688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.351799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1ff6-6266-4dfe-9531-04e289a34796 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.357319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.357961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.358240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.358513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.359197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.361320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.377802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:00.399633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1290/2518] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:00 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:00.404122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.415095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b66f9-5cb8-46d0-aae6-bd713a64fefe admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.470416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:00.482792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.483507 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.484348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.484636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.485308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.487306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.512707 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1229/2519] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:00 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:00.517217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.530947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f355e76-6c4a-41be-9e18-96893cd84ea7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.538927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.541651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.541929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.542650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.543373 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.545102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.559748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1291/2520] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:00.561358 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.573991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93528ee9-67f1-46c5-a277-12e31fd92436 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.581007 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c1fe8-d27a-435b-8cda-6f29ca5dbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.581648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c1fe8-d27a-435b-8cda-6f29ca5dbc17 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.581927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c1fe8-d27a-435b-8cda-6f29ca5dbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.582224 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c1fe8-d27a-435b-8cda-6f29ca5dbc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.582978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c1fe8-d27a-435b-8cda-6f29ca5dbc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.585384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c1fe8-d27a-435b-8cda-6f29ca5dbc17 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.598379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1230/2521] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:00.603590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.614923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73523151-33ef-44fb-80d6-e0df3602337e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.620092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.620790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.621051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.621326 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] PATH_INFO: `/v3/projects/3e7faeff1e5d4cdf9ad000fa6d756310/users/38ca26881d0a401ba725ba269f640902/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.638914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7faeff1e5d4cdf9ad000fa6d756310, user_id=38ca26881d0a401ba725ba269f640902, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.641197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.659324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1292/2522] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:00 2022] PUT /identity/v3/projects/3e7faeff1e5d4cdf9ad000fa6d756310/users/38ca26881d0a401ba725ba269f640902/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:00.670782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.671071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.671415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.705524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] MFA Rules not processed for user `38ca26881d0a401ba725ba269f640902`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:00.707557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:00.792657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1231/2523] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:00.804555 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:00.806602 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:00.893735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b9f0-ae98-447f-a4a2-802b30050c17 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:00.900986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:00.901634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:00.901910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:00.902211 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:00.911330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:00.913359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:00.924032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1293/2524] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:07.146830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:07.221475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e135895-3a0c-412d-ba5b-794d5921c5a1 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:07.228567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:07.229397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.229976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.230373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] PATH_INFO: `/v3/users/38ca26881d0a401ba725ba269f640902` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.232566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ca26881d0a401ba725ba269f640902)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:07.234808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:07.236528 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:07.239365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.264302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598b60-9128-4de7-9189-f5060b3d0391 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:07.270642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:07.271518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.272110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.272605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] PATH_INFO: `/v3/users/888d9c44e51c428aab0ae7ac78d307b0` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.274761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888d9c44e51c428aab0ae7ac78d307b0)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:07.278615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:07.299890 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ca26881d0a401ba725ba269f640902'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.300603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ca26881d0a401ba725ba269f640902'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.319306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:07.323044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ca26881d0a401ba725ba269f640902'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.339320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ca26881d0a401ba725ba269f640902'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.340327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-89a20b9d-dd86-4557-9318-2449370dee24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ca26881d0a401ba725ba269f640902'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.383200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1232/2525] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:07 2022] DELETE /identity/v3/users/38ca26881d0a401ba725ba269f640902 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:07.384900 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c380-f24c-42b0-ae5a-5cf59350e422 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.385323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c380-f24c-42b0-ae5a-5cf59350e422 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.385623 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c380-f24c-42b0-ae5a-5cf59350e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.387977 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888d9c44e51c428aab0ae7ac78d307b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.388754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888d9c44e51c428aab0ae7ac78d307b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.403613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888d9c44e51c428aab0ae7ac78d307b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.410120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2b96c380-f24c-42b0-ae5a-5cf59350e422 None None] MFA Rules not processed for user `5a83ef8e97ae4500943dfbcdd3fb7587`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:07.412410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2b96c380-f24c-42b0-ae5a-5cf59350e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.422179 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888d9c44e51c428aab0ae7ac78d307b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.430818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-6d10a60c-5b57-4393-88b4-657111402f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888d9c44e51c428aab0ae7ac78d307b0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:07.437322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1294/2526] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:07 2022] DELETE /identity/v3/users/888d9c44e51c428aab0ae7ac78d307b0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:46:07.442466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.442466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.442466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.465632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:07.467509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.483009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:07.487990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1233/2527] 10.222.0.45 () {58 vars in 967 bytes} [Mon Aug 1 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:07.490430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe932b-df38-46cf-8252-16779def419f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.490674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe932b-df38-46cf-8252-16779def419f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.491060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe932b-df38-46cf-8252-16779def419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.523240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-0efe932b-df38-46cf-8252-16779def419f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:07.524518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0efe932b-df38-46cf-8252-16779def419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.555740 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1295/2528] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:07.556992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:07.565290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.602471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1234/2529] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:07.603469 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe932b-df38-46cf-8252-16779def419f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:07.604797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0efe932b-df38-46cf-8252-16779def419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.677811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe932b-df38-46cf-8252-16779def419f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:07.683484 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd44244-ba29-40a3-879a-1934ec0db267 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:07.687061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:07.687727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.688033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.688819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.696654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4177b0f6e8204e5285ee4fdaeb2aa4c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:07.698698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.699188 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.699674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.700728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:07.702502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:07.714125 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1235/2530] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:07.715111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:07.716750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:07.729966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaf54e-240c-4bec-8b6c-8c73b0d6281f service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:07.735408 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:07.736089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:07.736557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:07.736831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:07.751263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:07.753321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:07.769344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1236/2531] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:07.790584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:07.792420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:07.812597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1296/2532] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Aug 1 16:46:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:46:08.251518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.254326 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:08.283418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09114-4e06-41b6-8d1f-947d7058a293 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.303295 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.304662 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.305248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.307949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] PATH_INFO: `/v3/projects/3e7faeff1e5d4cdf9ad000fa6d756310` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.311921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e7faeff1e5d4cdf9ad000fa6d756310)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.314006 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.334409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:08.377131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.380317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:08.401690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd71ad-05c6-4237-bce6-670d20244cbd service swift] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.412019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.412693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.412973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.413427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] PATH_INFO: `/v3/projects/e012f877d0da438ba9e2433436cc691a` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.415241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e012f877d0da438ba9e2433436cc691a)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.418296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.451476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e7faeff1e5d4cdf9ad000fa6d756310', 'request_id': 'req-c12e185d-8e36-46bb-92d2-2ac1348afae8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.458862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e7faeff1e5d4cdf9ad000fa6d756310', 'request_id': 'req-c12e185d-8e36-46bb-92d2-2ac1348afae8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.471947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e7faeff1e5d4cdf9ad000fa6d756310', 'request_id': 'req-c12e185d-8e36-46bb-92d2-2ac1348afae8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.493072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] The token cache is being invalidate because project 3e7faeff1e5d4cdf9ad000fa6d756310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:08.493820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.495042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c12e185d-8e36-46bb-92d2-2ac1348afae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.497481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1237/2533] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:08 2022] DELETE /identity/v3/projects/3e7faeff1e5d4cdf9ad000fa6d756310 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:46:08.503813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412d914-3620-4baa-a633-5214ff2d4918 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.504321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412d914-3620-4baa-a633-5214ff2d4918 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.504725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412d914-3620-4baa-a633-5214ff2d4918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.508539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e012f877d0da438ba9e2433436cc691a', 'request_id': 'req-61d4cee1-a3ec-446d-bd97-36d25848907c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.512449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:08.524212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e012f877d0da438ba9e2433436cc691a', 'request_id': 'req-61d4cee1-a3ec-446d-bd97-36d25848907c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.534616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e012f877d0da438ba9e2433436cc691a', 'request_id': 'req-61d4cee1-a3ec-446d-bd97-36d25848907c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.538777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a412d914-3620-4baa-a633-5214ff2d4918 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:08.541423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a412d914-3620-4baa-a633-5214ff2d4918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:08.550229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] The token cache is being invalidate because project e012f877d0da438ba9e2433436cc691a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:08.550575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.551410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-61d4cee1-a3ec-446d-bd97-36d25848907c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:08.553787 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1297/2534] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:08 2022] DELETE /identity/v3/projects/e012f877d0da438ba9e2433436cc691a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:08.560044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb98d9-fb16-49c0-9ed2-6f3ab6789bb3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.560319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb98d9-fb16-49c0-9ed2-6f3ab6789bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.560575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb98d9-fb16-49c0-9ed2-6f3ab6789bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.585020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-d7bb98d9-fb16-49c0-9ed2-6f3ab6789bb3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:08.587340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d7bb98d9-fb16-49c0-9ed2-6f3ab6789bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:08.653852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1238/2535] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:08.666273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.672098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.672098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.707532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:08.709382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1298/2536] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:08.712344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:08.719376 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.719710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.719985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.724516 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1239/2537] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:08.736749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.747856 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:08.748391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678db19-5060-4d6c-8d33-b77b47e1ddb2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.749441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:08.753985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.754862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.755249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.755555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.756501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:08.756871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.759145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.761985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1299/2538] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:08.766710 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.774334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1240/2539] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:46:08.779080 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.784246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aa6af-6d5d-4688-af8b-1ad8ad3f6246 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.792384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae0f2-bbf4-4f35-97d4-fab7927a44d9 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.793035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.793819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.794443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.794839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.795826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:08.796924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.798089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.798735 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.799013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.799284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.799441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.800180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.802799 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.825441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1300/2540] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:46:08.831720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.843552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1241/2541] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:08.849414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bc597-33bf-439a-bf46-3352799edebe admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.850759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.924278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:08.929465 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c57c5-065d-47c3-b88f-7f5d1d24484e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.936732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.938049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.938377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.939412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.940475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.942419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:08.943180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:08.943488 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:08.945725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.946943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:08.946943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:08.946943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:08.973591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1242/2542] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:08.979489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:08.994308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d8cb7-25b6-4e05-b280-a6f7082aa40f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:08.998070 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1301/2543] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:08 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:09.003059 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.062797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:09.073381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.074064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.074470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.074748 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.075848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.078203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.087100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4591-e3dd-485d-83c5-8ad25335f538 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.093945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1243/2544] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:09.094203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.094689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.094863 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.095085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.095789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.097825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.098009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.109656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e834d-de73-4ef7-b07f-bca90d40948f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.115714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.116422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.116705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.116979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.117704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.119411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.131926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1244/2545] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:09.136593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.139007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1302/2546] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:09 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:09.143947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.147917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf300a0-1978-4d47-99a9-e981dce2ce47 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.153528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45323a7f-608c-4190-820d-f81d9e366524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.154281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323a7f-608c-4190-820d-f81d9e366524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.154620 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323a7f-608c-4190-820d-f81d9e366524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.154814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323a7f-608c-4190-820d-f81d9e366524 admin admin] PATH_INFO: `/v3/projects/f712845306724b3a98ecc26aa4ce1d4f/users/410edf8cb68444cd9df7742990d25cb6/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.156737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4fc49-7b2e-45a6-9313-936e148df8df admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.163611 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.164522 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.164975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.165337 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.166146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.168004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.172575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45323a7f-608c-4190-820d-f81d9e366524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f712845306724b3a98ecc26aa4ce1d4f, user_id=410edf8cb68444cd9df7742990d25cb6, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.175348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45323a7f-608c-4190-820d-f81d9e366524 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.181321 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1303/2547] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:09.186357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.197402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3baa2-7eb6-4232-9049-c03b02453714 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.200313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1245/2548] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:09 2022] PUT /identity/v3/projects/f712845306724b3a98ecc26aa4ce1d4f/users/410edf8cb68444cd9df7742990d25cb6/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:09.212563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.212853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.213189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.251038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] MFA Rules not processed for user `410edf8cb68444cd9df7742990d25cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:09.252807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:09.295065 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.295704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.296006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.296277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.297390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.298607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.309656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1304/2549] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:09.314012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.325629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d2154-0d07-4bd7-9cc0-2f767556e22b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.331242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a644e4-c142-45ce-9af6-9d18110b971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.331981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a644e4-c142-45ce-9af6-9d18110b971c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.332278 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a644e4-c142-45ce-9af6-9d18110b971c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.333014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a644e4-c142-45ce-9af6-9d18110b971c admin admin] PATH_INFO: `/v3/projects/b0e048a61c324891a03130fff8bda0b1/users/03880809413c43629852d3f9dc865d32/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.340706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1246/2550] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:09.347346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.353213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a644e4-c142-45ce-9af6-9d18110b971c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e048a61c324891a03130fff8bda0b1, user_id=03880809413c43629852d3f9dc865d32, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.355981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a644e4-c142-45ce-9af6-9d18110b971c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.359106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd9c51-8d02-436d-949f-3a06b95daf01 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.364694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.365618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.365967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.366241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.366921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.368952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.383898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1305/2551] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:09 2022] PUT /identity/v3/projects/b0e048a61c324891a03130fff8bda0b1/users/03880809413c43629852d3f9dc865d32/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:09.390233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.390530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.390830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.413289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1247/2552] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:09 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:09.414895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.431964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] MFA Rules not processed for user `03880809413c43629852d3f9dc865d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:09.434498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:09.451346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:09.508481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3078a-21a9-456a-b202-71449268cf2a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.513287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.513922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.514196 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.514460 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.515140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.517111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.522654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1306/2553] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:09.535594 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.537658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:09.546441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1248/2554] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:09 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:09.550188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.560634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2659ca-8b85-4914-9695-fd604f5fb97b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.565607 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.566393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.566678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.567059 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.567793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.569794 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.586444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1249/2555] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:09.587919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.602316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43012414-32c5-4b88-9731-e7f573935822 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.605462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:09.607670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.608454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.608722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.609065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] PATH_INFO: `/v3/projects/0f7a16026e9d4ae287b40f13b7cbdf8f/users/88be589336204348813d0afcfa0f7e99/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.634243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7a16026e9d4ae287b40f13b7cbdf8f, user_id=88be589336204348813d0afcfa0f7e99, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.637256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.647330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3410d-284f-4694-ad73-9e184ac1cb93 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.651830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.652397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.652674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.652991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.663458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.664997 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.665636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1250/2556] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:09 2022] PUT /identity/v3/projects/0f7a16026e9d4ae287b40f13b7cbdf8f/users/88be589336204348813d0afcfa0f7e99/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:09.671805 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.676334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1307/2557] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:09.773240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7823be-1c1f-475b-a1bf-083f83bee732 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.780611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff0547-1d0e-4a5f-9a3f-24b564c933b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.781377 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0547-1d0e-4a5f-9a3f-24b564c933b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.781709 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0547-1d0e-4a5f-9a3f-24b564c933b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.782146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff0547-1d0e-4a5f-9a3f-24b564c933b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.783566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff0547-1d0e-4a5f-9a3f-24b564c933b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.786157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff0547-1d0e-4a5f-9a3f-24b564c933b7 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.807605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1251/2558] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:09.809117 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:09.822539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a40b-d610-406c-a106-2ef2eaa44fd1 service glance] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:09.827804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:09.828485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.828763 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.829029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] PATH_INFO: `/v3/projects/0f7a16026e9d4ae287b40f13b7cbdf8f/users/88be589336204348813d0afcfa0f7e99/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.831527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7a16026e9d4ae287b40f13b7cbdf8f, user_id=88be589336204348813d0afcfa0f7e99, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:09.834820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:09.851477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1308/2559] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:09 2022] PUT /identity/v3/projects/0f7a16026e9d4ae287b40f13b7cbdf8f/users/88be589336204348813d0afcfa0f7e99/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:09.866315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:09.866819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:09.867211 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:09.909248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] MFA Rules not processed for user `88be589336204348813d0afcfa0f7e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:09.911016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:09.985262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:10.014563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1252/2560] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:10.027969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:10.029923 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:10.106970 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24f89-7d77-49f9-98b9-5d0ec7903359 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:10.110790 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:10.111411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:10.111684 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:10.112099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:10.122482 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:10.124033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:10.134832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1309/2561] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:46:10.237521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:10.239314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:10.262314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f66594-fe86-406b-a062-df0c6e039f5f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:10.266726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f97b5-79cb-43ea-ba49-baa4721ff12d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:10.267546 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f97b5-79cb-43ea-ba49-baa4721ff12d service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:10.267829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f97b5-79cb-43ea-ba49-baa4721ff12d service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:10.268103 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f97b5-79cb-43ea-ba49-baa4721ff12d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:10.339864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f97b5-79cb-43ea-ba49-baa4721ff12d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:10.341310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f97b5-79cb-43ea-ba49-baa4721ff12d service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:10.351928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1253/2562] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:10.752721 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:10.755131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:10.852186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de49c3-c7ca-42fd-9bd7-10528c82c772 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:10.857531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:10.858289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:10.858539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:10.858822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] PATH_INFO: `/v3/users/8aa4ce0580fa4b3ba8eeb59dbb06e912` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:10.860272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa4ce0580fa4b3ba8eeb59dbb06e912)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:10.869788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:10.928177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa4ce0580fa4b3ba8eeb59dbb06e912'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:10.928939 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa4ce0580fa4b3ba8eeb59dbb06e912'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:10.939547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa4ce0580fa4b3ba8eeb59dbb06e912'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:10.948574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:10.960717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa4ce0580fa4b3ba8eeb59dbb06e912'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:10.969170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa4ce0580fa4b3ba8eeb59dbb06e912'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:10.976165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1310/2563] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:10 2022] DELETE /identity/v3/users/8aa4ce0580fa4b3ba8eeb59dbb06e912 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:10.981179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:10.981788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:10.981788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:11.011810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:11.013148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:11.092021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1254/2564] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:11.102462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:11.104075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:11.174676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c26a4-1a05-42c4-89b2-5e17217c9077 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:11.178454 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:11.179075 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:11.179351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:11.179619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:11.190137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:11.191685 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:11.208981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1311/2565] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:11.519429 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:11.521142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:11.538205 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a167fb-2a46-4115-a976-5aed28100197 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:11.545578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:11.546079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:11.546378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:11.546681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] PATH_INFO: `/v3/projects/b2dadcbe5b4a43f4b72b103682476a94` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:11.548337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2dadcbe5b4a43f4b72b103682476a94)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:11.550673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:11.618608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2dadcbe5b4a43f4b72b103682476a94', 'request_id': 'req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.636107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2dadcbe5b4a43f4b72b103682476a94', 'request_id': 'req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.649540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2dadcbe5b4a43f4b72b103682476a94', 'request_id': 'req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.675909 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] The token cache is being invalidate because project b2dadcbe5b4a43f4b72b103682476a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:11.676838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.677772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.679553 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1255/2566] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:11 2022] DELETE /identity/v3/projects/b2dadcbe5b4a43f4b72b103682476a94 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:11.683969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:11.696640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea88757-a1db-4ffc-8cea-8e71ad2c57ba service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:11.774324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:11.775196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:11.775309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:11.775599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] PATH_INFO: `/v3/users/0e4848058c1145fbac9ce4222e7a1fcd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:11.776809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4848058c1145fbac9ce4222e7a1fcd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:11.778868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:11.849037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4848058c1145fbac9ce4222e7a1fcd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.850074 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4848058c1145fbac9ce4222e7a1fcd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.865502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4848058c1145fbac9ce4222e7a1fcd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.886947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4848058c1145fbac9ce4222e7a1fcd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.894577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-21fe9b7e-be8d-4064-97dd-32ef4591157a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4848058c1145fbac9ce4222e7a1fcd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:11.897633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:11.903468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1312/2567] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:11 2022] DELETE /identity/v3/users/0e4848058c1145fbac9ce4222e7a1fcd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:12.241997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:12.248232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:12.278819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779bfe-d33a-4bbb-a5c6-89820ac892a4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:12.292241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:12.413145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:12.414316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:12.414952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:12.415309 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] PATH_INFO: `/v3/projects/c22d9fb40fda4312900b14eda8bf1526` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:12.417734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22d9fb40fda4312900b14eda8bf1526)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:12.421586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:12.512175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22d9fb40fda4312900b14eda8bf1526', 'request_id': 'req-17266250-cc32-433c-9b5a-482dbfc23b57'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:12.518866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22d9fb40fda4312900b14eda8bf1526', 'request_id': 'req-17266250-cc32-433c-9b5a-482dbfc23b57'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:12.529643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22d9fb40fda4312900b14eda8bf1526', 'request_id': 'req-17266250-cc32-433c-9b5a-482dbfc23b57'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:12.544290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] The token cache is being invalidate because project c22d9fb40fda4312900b14eda8bf1526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:12.544577 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:12.545231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-17266250-cc32-433c-9b5a-482dbfc23b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:12.546833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1256/2568] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:12 2022] DELETE /identity/v3/projects/c22d9fb40fda4312900b14eda8bf1526 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:12.552263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:12.552559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:12.552927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:12.578419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:12.580081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:12.669319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1313/2569] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:12.678772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:12.679064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:12.679335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:12.700605 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:12.702149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:12.712621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1257/2570] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:12.717642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:12.729711 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46073c-f2be-4300-a02e-9c4ee9165a3d None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:12.735989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:12.737014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:12.737907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:12.737907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:12.738676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:12.739877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:12.741913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:12.757349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1314/2571] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:12.770858 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:12.787294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bf551-c546-4e93-a1f0-f53a44deaa91 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:12.795235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:12.796422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:12.796935 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:12.797509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:12.798868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:12.802936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:12.844762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1258/2572] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:12 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:12.849916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:12.867405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e503-ca5d-4c95-bc31-75b6c3f3c890 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.117569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.118571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.118855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.119127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.120545 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.122544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.151552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1315/2573] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:12 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:13.156005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.177592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd9cb2-4fd5-4dcf-87ae-0d0f34bc6cec admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.189388 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.190341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.190341 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.190668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.191225 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.193752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.214177 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1259/2574] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:13.216252 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.232157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900bc35-1761-4ecf-8f0d-54b7992c67b7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.237266 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2a5cd-92ef-4d22-9083-a9b2ebed6b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.238064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2a5cd-92ef-4d22-9083-a9b2ebed6b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.238446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2a5cd-92ef-4d22-9083-a9b2ebed6b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.238804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2a5cd-92ef-4d22-9083-a9b2ebed6b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.240248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2a5cd-92ef-4d22-9083-a9b2ebed6b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.242094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2a5cd-92ef-4d22-9083-a9b2ebed6b95 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.259806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1316/2575] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:13.262996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.275646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ff9ed-0174-4af8-aabe-cb9fc3464600 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.282272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.283570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.288356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.288713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] PATH_INFO: `/v3/projects/68818efd24a0494584057c4d2d159d72/users/91bb1d17c8ea4f0094bcd357ad80777e/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.309230 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68818efd24a0494584057c4d2d159d72, user_id=91bb1d17c8ea4f0094bcd357ad80777e, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.311885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.335941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1260/2576] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:13 2022] PUT /identity/v3/projects/68818efd24a0494584057c4d2d159d72/users/91bb1d17c8ea4f0094bcd357ad80777e/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:13.350972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.351260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.351713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.427906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] MFA Rules not processed for user `91bb1d17c8ea4f0094bcd357ad80777e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:13.429150 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:13.546885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:13.558636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1317/2577] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:13.562996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.651242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:13.686876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ab9d4-eb7d-4a10-a278-454fa95ac46a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.696290 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.697089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.697525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.697804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.698494 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.700578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.739409 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1261/2578] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:13 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:13.743651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.757216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61a582-e012-43ce-966b-bae8f7d6a443 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.829217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.829781 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.830028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.830295 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.830991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.833116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.862414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1318/2579] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:13 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:13.863896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.874645 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb8e2-f415-47eb-bd2a-23b9c7955016 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.879618 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.880246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.880521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.880826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.881492 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.883139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.898993 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1262/2580] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:13.903432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.914186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e5c19-b1df-40f9-839d-accacc8719ab admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:13.919275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:13.920338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:13.920623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:13.920898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] PATH_INFO: `/v3/projects/619e7ce6f2884e3e9d2730c4bb97bd8c/users/d46c741cd3c14bf4b90a7f80928104b9/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:13.940559 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619e7ce6f2884e3e9d2730c4bb97bd8c, user_id=d46c741cd3c14bf4b90a7f80928104b9, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:13.943432 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:13.962391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1319/2581] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:13 2022] PUT /identity/v3/projects/619e7ce6f2884e3e9d2730c4bb97bd8c/users/d46c741cd3c14bf4b90a7f80928104b9/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:13.966449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:13.978982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34888bc-92fa-44bd-a134-942a09948e6e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:14.062013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85500ef1-60d4-482b-95fd-9b0c6c48cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:14.062655 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500ef1-60d4-482b-95fd-9b0c6c48cf64 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:14.064022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500ef1-60d4-482b-95fd-9b0c6c48cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:14.064022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85500ef1-60d4-482b-95fd-9b0c6c48cf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:14.064022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85500ef1-60d4-482b-95fd-9b0c6c48cf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:14.065532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85500ef1-60d4-482b-95fd-9b0c6c48cf64 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:14.079141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1263/2582] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:14.083915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:14.097355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8c201-82b2-4b01-9d33-5dc525b17546 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:14.103427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:14.104309 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:14.104806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:14.105194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] PATH_INFO: `/v3/projects/619e7ce6f2884e3e9d2730c4bb97bd8c/users/d46c741cd3c14bf4b90a7f80928104b9/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:14.107655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619e7ce6f2884e3e9d2730c4bb97bd8c, user_id=d46c741cd3c14bf4b90a7f80928104b9, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:14.110938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:14.132394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1320/2583] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:14 2022] PUT /identity/v3/projects/619e7ce6f2884e3e9d2730c4bb97bd8c/users/d46c741cd3c14bf4b90a7f80928104b9/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:14.207603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126107bd-4d27-495c-be16-1263c77a8a80 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:14.208156 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126107bd-4d27-495c-be16-1263c77a8a80 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:14.208667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126107bd-4d27-495c-be16-1263c77a8a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:14.247128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-126107bd-4d27-495c-be16-1263c77a8a80 None None] MFA Rules not processed for user `d46c741cd3c14bf4b90a7f80928104b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:14.248464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-126107bd-4d27-495c-be16-1263c77a8a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:14.358927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1264/2584] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:14.371261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:14.373435 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:14.460223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319775-205b-4b43-be18-fd9c24036ed0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:14.464628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:14.465348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:14.465724 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:14.466096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:14.539290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:14.543177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:14.560699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1321/2585] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:18.083223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:18.083726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:18.083841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:18.104416 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] MFA Rules not processed for user `22cfd17a6f434deca7ef77253ae3ed2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:18.105669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:18.140959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:18.170658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1265/2586] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:18.534156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:18.535953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:18.541960 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:18.608495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d67dd-104a-4340-a00c-29ac9b16d28b service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:18.612243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:18.612874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:18.613149 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:18.613639 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:18.615569 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:18.625800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:18.627325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:18.637691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1322/2587] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:20.037010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:20.039076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:20.153726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521036c-20d4-418c-95d1-f8dc82da7180 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:20.167542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:20.168274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:20.169089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:20.169089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] PATH_INFO: `/v3/users/410edf8cb68444cd9df7742990d25cb6` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:20.170185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=410edf8cb68444cd9df7742990d25cb6)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:20.172564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:20.236235 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '410edf8cb68444cd9df7742990d25cb6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:20.237810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '410edf8cb68444cd9df7742990d25cb6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:20.267833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '410edf8cb68444cd9df7742990d25cb6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:20.277817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '410edf8cb68444cd9df7742990d25cb6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:20.278583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-fef12bd6-922d-4709-bf7b-6f03ef8734fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '410edf8cb68444cd9df7742990d25cb6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:20.287414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:20.291092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:20.302400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1266/2588] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:20 2022] DELETE /identity/v3/users/410edf8cb68444cd9df7742990d25cb6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:20.307180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:20.307514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:20.307812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:20.309586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e813a-3c94-48eb-8146-d1fd6c365a29 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:20.332593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:20.334471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:20.417027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:20.418082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:20.418577 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:20.419195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:20.421424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:20.444413 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:20.469915 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1267/2589] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:20.480180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:20.487029 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:20.524679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:20.526421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:20.544302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1323/2590] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:20.594147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76987af-43a5-41a0-acdc-2f660a2fb82b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:20.599721 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:20.600635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:20.600880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:20.601175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:20.619597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:20.621139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:20.634436 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1268/2591] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:21.140868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:21.142598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:21.163042 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ed876-a3e2-4477-a3be-3b5c2e20e196 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:21.175415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:21.179981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:21.181539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:21.181898 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] PATH_INFO: `/v3/projects/f712845306724b3a98ecc26aa4ce1d4f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:21.184686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f712845306724b3a98ecc26aa4ce1d4f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:21.188445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:21.273595 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f712845306724b3a98ecc26aa4ce1d4f', 'request_id': 'req-f72fc309-7330-48be-b233-b5c9ef84d1cd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.275032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:21.283249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f712845306724b3a98ecc26aa4ce1d4f', 'request_id': 'req-f72fc309-7330-48be-b233-b5c9ef84d1cd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.299803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f712845306724b3a98ecc26aa4ce1d4f', 'request_id': 'req-f72fc309-7330-48be-b233-b5c9ef84d1cd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.316319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] The token cache is being invalidate because project f712845306724b3a98ecc26aa4ce1d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:21.316813 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.317530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.319245 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1324/2592] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:21 2022] DELETE /identity/v3/projects/f712845306724b3a98ecc26aa4ce1d4f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:21.323394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:21.334559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4cbd2-09da-489b-8ccd-69a0df0015df service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:21.409030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:21.409791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:21.410181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:21.410457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] PATH_INFO: `/v3/users/88be589336204348813d0afcfa0f7e99` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:21.411775 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88be589336204348813d0afcfa0f7e99)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:21.413783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:21.504661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88be589336204348813d0afcfa0f7e99'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.504661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88be589336204348813d0afcfa0f7e99'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.523261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88be589336204348813d0afcfa0f7e99'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.537986 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88be589336204348813d0afcfa0f7e99'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.538750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-a5d9b93b-a0f5-4d35-ae75-f596ff5eec95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88be589336204348813d0afcfa0f7e99'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:21.557957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1269/2593] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:21 2022] DELETE /identity/v3/users/88be589336204348813d0afcfa0f7e99 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:22.128120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:22.131016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:22.207355 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fc309-7330-48be-b233-b5c9ef84d1cd admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:22.320220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:22.321297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.321297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.321514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] PATH_INFO: `/v3/projects/0f7a16026e9d4ae287b40f13b7cbdf8f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.322683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7a16026e9d4ae287b40f13b7cbdf8f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:22.325091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:22.398571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7a16026e9d4ae287b40f13b7cbdf8f', 'request_id': 'req-84f0c4e8-15ac-423f-912c-7e1507e4b324'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:22.405915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7a16026e9d4ae287b40f13b7cbdf8f', 'request_id': 'req-84f0c4e8-15ac-423f-912c-7e1507e4b324'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:22.416905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7a16026e9d4ae287b40f13b7cbdf8f', 'request_id': 'req-84f0c4e8-15ac-423f-912c-7e1507e4b324'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:22.431014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] The token cache is being invalidate because project 0f7a16026e9d4ae287b40f13b7cbdf8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:22.431281 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:22.432007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-84f0c4e8-15ac-423f-912c-7e1507e4b324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:22.433706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1325/2594] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:22 2022] DELETE /identity/v3/projects/0f7a16026e9d4ae287b40f13b7cbdf8f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:46:22.438940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.439475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.439475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.465524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:22.467168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:22.490933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:22.551864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1270/2595] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:22.562253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.562877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.563187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.585160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:22.587100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:22.595129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1326/2596] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:22.599696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:22.609848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477bd3-0ef4-4321-99ee-76b0d1a9a05a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:22.616105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:22.617041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.617537 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.618031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.618958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:22.619474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:22.621438 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:22.636275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1271/2597] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:22.640907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:22.651001 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bd151-98bb-4487-b302-91e4c3f54356 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:22.656035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:22.656680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.656951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.657243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.657913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:22.659864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:22.698323 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:22.699126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1327/2598] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:22 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:22.711383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e10c4-20f4-491d-a678-db0dbd91b26b admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:22.919071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:22.919864 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.920139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.920410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.921073 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:22.923076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:22.955028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1272/2599] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:22 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:22.959311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:22.970553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc970a-361d-4f9f-9a2d-ed6124fae692 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:22.976598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:22.977467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:22.977819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:22.978096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:22.979815 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:22.981681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.000060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1328/2600] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:23.004169 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.014222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a9a04-c663-4e99-a478-95a43bd9c706 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.019241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc27861-8aed-4fcc-9c21-caa9a3cb26f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.020031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc27861-8aed-4fcc-9c21-caa9a3cb26f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.020389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc27861-8aed-4fcc-9c21-caa9a3cb26f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.020828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc27861-8aed-4fcc-9c21-caa9a3cb26f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.021597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc27861-8aed-4fcc-9c21-caa9a3cb26f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.023275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc27861-8aed-4fcc-9c21-caa9a3cb26f5 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.034352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1273/2601] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:23.045092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.064177 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc3c13-6d08-403c-9d38-26be26e41817 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.071803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.072833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.073180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.073541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] PATH_INFO: `/v3/projects/c60c99ff3e5a4690bcb93b4c08bb2b2b/users/f50c5fa1fa9041a2b1e1c3ae1022ba1c/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.103499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60c99ff3e5a4690bcb93b4c08bb2b2b, user_id=f50c5fa1fa9041a2b1e1c3ae1022ba1c, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.106525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.135100 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1329/2602] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:23 2022] PUT /identity/v3/projects/c60c99ff3e5a4690bcb93b4c08bb2b2b/users/f50c5fa1fa9041a2b1e1c3ae1022ba1c/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:23.145473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.146031 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.146292 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.175441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] MFA Rules not processed for user `f50c5fa1fa9041a2b1e1c3ae1022ba1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:23.176692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:23.246990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1274/2603] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:23.251742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.258311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:23.322689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a142033-394a-4c30-ac15-968e757195ca admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.327714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.328365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.328640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.328900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.329599 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.331515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.361453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1330/2604] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:23 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:23.365996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.380338 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec668e2d-6a38-4e4f-876f-912e102b5aae None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.395723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:23.442817 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.443475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.443782 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.444065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.444747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.446663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.470613 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1275/2605] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:23 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:23.474845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.487912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476f20-3453-486d-a3db-344422bb13c4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.495083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.495827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.496189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.496530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.497414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.500236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.515115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1331/2606] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:23.519277 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.533558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8682-9bd3-4865-ba26-a2b28e764edc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.539411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.540269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.540738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.541201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] PATH_INFO: `/v3/projects/9b97eea37ba84758a977aaa4dda47ed7/users/f38569b77c9d4bfb82f50ea039f409dd/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.559610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b97eea37ba84758a977aaa4dda47ed7, user_id=f38569b77c9d4bfb82f50ea039f409dd, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.562970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.590160 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1276/2607] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:23 2022] PUT /identity/v3/projects/9b97eea37ba84758a977aaa4dda47ed7/users/f38569b77c9d4bfb82f50ea039f409dd/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:23.593982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.606240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1b02e-5cc4-474c-92d8-a4408994efe2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.685348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93af97-4193-4bfb-9cb1-75fba8f3261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.685983 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93af97-4193-4bfb-9cb1-75fba8f3261d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.686299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93af97-4193-4bfb-9cb1-75fba8f3261d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.686524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93af97-4193-4bfb-9cb1-75fba8f3261d admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.687244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93af97-4193-4bfb-9cb1-75fba8f3261d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.688885 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93af97-4193-4bfb-9cb1-75fba8f3261d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.699473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1332/2608] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:23.703873 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.716161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c78126-4535-40c5-bd07-7693be441a75 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.721231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.722201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.722342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.722610 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] PATH_INFO: `/v3/projects/9b97eea37ba84758a977aaa4dda47ed7/users/f38569b77c9d4bfb82f50ea039f409dd/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.724996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b97eea37ba84758a977aaa4dda47ed7, user_id=f38569b77c9d4bfb82f50ea039f409dd, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.728045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.740048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1277/2609] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:23 2022] PUT /identity/v3/projects/9b97eea37ba84758a977aaa4dda47ed7/users/f38569b77c9d4bfb82f50ea039f409dd/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:46:23.752324 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.752591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.752868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.786352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] MFA Rules not processed for user `f38569b77c9d4bfb82f50ea039f409dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:23.787744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:23.864512 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:23.868414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1333/2610] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:23.879829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:23.881538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:23.938963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282803-d82d-453e-b795-8379c113e87a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:23.942877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:23.943472 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:23.943745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:23.944042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:23.953391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:23.954870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:23.965107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1278/2611] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:24.013421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:24.014915 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:24.028073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514d01-de15-4b6f-9500-98cf3bf355e8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:24.032043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2c558-477b-4b86-a38c-f65ce7e29689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:24.032647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2c558-477b-4b86-a38c-f65ce7e29689 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:24.032930 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2c558-477b-4b86-a38c-f65ce7e29689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:24.033202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2c558-477b-4b86-a38c-f65ce7e29689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:24.093406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2c558-477b-4b86-a38c-f65ce7e29689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:24.094857 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2c558-477b-4b86-a38c-f65ce7e29689 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:24.104427 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1334/2612] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:25.815143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:25.817220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:25.890679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b630ca6-d6a4-4cf5-9df5-3c34e313ce46 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:25.895950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:25.896611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:25.896870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:25.897145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] PATH_INFO: `/v3/users/91bb1d17c8ea4f0094bcd357ad80777e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:25.898315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91bb1d17c8ea4f0094bcd357ad80777e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:25.900422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:25.915154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:25.960289 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91bb1d17c8ea4f0094bcd357ad80777e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:25.961532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91bb1d17c8ea4f0094bcd357ad80777e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:25.977131 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91bb1d17c8ea4f0094bcd357ad80777e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:25.984617 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91bb1d17c8ea4f0094bcd357ad80777e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:25.985364 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91bb1d17c8ea4f0094bcd357ad80777e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.002242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1279/2613] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:25 2022] DELETE /identity/v3/users/91bb1d17c8ea4f0094bcd357ad80777e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:26.006634 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:26.007191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:26.007290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:26.027006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:26.028286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:26.107605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1335/2614] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:26.119178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:26.121807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:26.199922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca097e-6c6b-4291-a294-2bd5e504592f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:26.203720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:26.204401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:26.204679 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:26.204957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:26.219344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:26.220862 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:26.231997 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1280/2615] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:26.575150 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:26.578540 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:26.606962 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7f9e8-ced6-4950-9641-7bed91f34451 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:26.612692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:26.614050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:26.614536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:26.614902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] PATH_INFO: `/v3/projects/68818efd24a0494584057c4d2d159d72` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:26.616394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] RBAC: Authorizing `identity:delete_project(project_id=68818efd24a0494584057c4d2d159d72)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:26.619018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:26.687191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68818efd24a0494584057c4d2d159d72', 'request_id': 'req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.695393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68818efd24a0494584057c4d2d159d72', 'request_id': 'req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.705617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68818efd24a0494584057c4d2d159d72', 'request_id': 'req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.722573 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] The token cache is being invalidate because project 68818efd24a0494584057c4d2d159d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:26.723871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.724648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.726746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1336/2616] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:26 2022] DELETE /identity/v3/projects/68818efd24a0494584057c4d2d159d72 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:26.731122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:26.742942 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c010768-6af7-4fcd-b0d5-288bb65047a6 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:26.816844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:26.817594 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:26.817937 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:26.818152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] PATH_INFO: `/v3/users/d46c741cd3c14bf4b90a7f80928104b9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:26.819533 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46c741cd3c14bf4b90a7f80928104b9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:26.821647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:26.872272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:26.884437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46c741cd3c14bf4b90a7f80928104b9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.885506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46c741cd3c14bf4b90a7f80928104b9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.904845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46c741cd3c14bf4b90a7f80928104b9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.914761 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46c741cd3c14bf4b90a7f80928104b9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.915536 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-409479e0-cbcf-4079-bf67-b3c6e40e7472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46c741cd3c14bf4b90a7f80928104b9'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:26.933063 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1281/2617] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:26 2022] DELETE /identity/v3/users/d46c741cd3c14bf4b90a7f80928104b9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:27.224757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.226263 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.242557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c1f7-79f5-4db4-82ca-85f9b6cb824e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.264136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:27.306834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.307515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.307780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.308067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] PATH_INFO: `/v3/projects/619e7ce6f2884e3e9d2730c4bb97bd8c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.309333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] RBAC: Authorizing `identity:delete_project(project_id=619e7ce6f2884e3e9d2730c4bb97bd8c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.311272 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.362203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619e7ce6f2884e3e9d2730c4bb97bd8c', 'request_id': 'req-2081a3fa-8aad-4514-af20-67198743767c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.368627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619e7ce6f2884e3e9d2730c4bb97bd8c', 'request_id': 'req-2081a3fa-8aad-4514-af20-67198743767c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.377544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619e7ce6f2884e3e9d2730c4bb97bd8c', 'request_id': 'req-2081a3fa-8aad-4514-af20-67198743767c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.392670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] The token cache is being invalidate because project 619e7ce6f2884e3e9d2730c4bb97bd8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:27.393111 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.393935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-2081a3fa-8aad-4514-af20-67198743767c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.395814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1337/2618] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:27 2022] DELETE /identity/v3/projects/619e7ce6f2884e3e9d2730c4bb97bd8c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:27.402299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.402663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.403014 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.424363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:27.425731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.492252 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1282/2619] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:27.501470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.501925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.502267 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.525644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:27.527618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.536354 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1338/2620] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:27.541111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.551079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96052ec0-4c90-4528-9e94-6bcebc450946 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.555786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.556417 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.556690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.556956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.557764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:27.557977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.559563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.571868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1283/2621] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:46:27.576619 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.578121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.584562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.586640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7e0de-db31-4b3f-a198-559a7b976f9f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.591317 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.591978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.592247 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.592515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.593176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.595055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.598697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7312a60-dddc-49a6-87fe-acca283ec413 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.603970 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.604622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.605360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.605639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] PATH_INFO: `/v3/users/f50c5fa1fa9041a2b1e1c3ae1022ba1c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.606821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50c5fa1fa9041a2b1e1c3ae1022ba1c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.608788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.623588 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1339/2622] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:27 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:27.632304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.669718 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50c5fa1fa9041a2b1e1c3ae1022ba1c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.671687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50c5fa1fa9041a2b1e1c3ae1022ba1c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.684725 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50c5fa1fa9041a2b1e1c3ae1022ba1c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.692046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50c5fa1fa9041a2b1e1c3ae1022ba1c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.692755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6bb52a47-8d34-43a5-a8e9-c29732fb9f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50c5fa1fa9041a2b1e1c3ae1022ba1c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:27.707658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1284/2623] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:27 2022] DELETE /identity/v3/users/f50c5fa1fa9041a2b1e1c3ae1022ba1c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:27.712258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.712567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.712834 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.716242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1ab87-f3fb-43f8-abb3-c511b585db5f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.733357 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:27.734677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.743689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:27.748841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:27.783891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.784525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.784799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.785118 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.785797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.787961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.818335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1340/2624] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:27 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:27.820448 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1285/2625] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:27.821365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.825053 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.830530 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.833071 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:27.840356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e60c78-aa22-4d4b-899a-1bbee3036c38 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.849463 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.850123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.850401 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.850714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.851430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.853759 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.867884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1341/2626] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:27.872071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.889902 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be22516-8bb4-4781-ae3d-7169e8b6a2d3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.896131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.897116 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.898107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.898277 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.899592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.901622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.909972 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fbbfa-397c-4c36-aee4-07cfd0af2d2c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.915328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1342/2627] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:27.919140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:27.919288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920feb-1fda-45db-a948-bb27188138c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.920025 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920feb-1fda-45db-a948-bb27188138c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.920497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920feb-1fda-45db-a948-bb27188138c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.920769 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920feb-1fda-45db-a948-bb27188138c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.935450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df38e1-4630-4831-9a35-cb1c25bb56a4 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:27.938215 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49920feb-1fda-45db-a948-bb27188138c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.939808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49920feb-1fda-45db-a948-bb27188138c0 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:27.950914 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1286/2628] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:27.951616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:27.952334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.952606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.952871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] PATH_INFO: `/v3/projects/5c995a4d9d2a4778b2d75214f1b35518/users/75feb70b6a51485aa62290467603a71c/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.975141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c995a4d9d2a4778b2d75214f1b35518, user_id=75feb70b6a51485aa62290467603a71c, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:27.977886 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:28.003943 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1343/2629] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:27 2022] PUT /identity/v3/projects/5c995a4d9d2a4778b2d75214f1b35518/users/75feb70b6a51485aa62290467603a71c/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:28.007179 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.007488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.007901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.041461 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] MFA Rules not processed for user `75feb70b6a51485aa62290467603a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:28.042829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.118482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1287/2630] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:28.130583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:28.132831 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.198833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4c705-4ad8-4bf5-81bd-81cd5e6c1b05 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:28.202750 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:28.203478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.203751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.204093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.213929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:28.215388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:28.225921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1344/2631] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:28.268578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:28.270231 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.342678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:28.363113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69be7e-4a08-4ab9-a417-c3daea1fc2af None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:28.368175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:28.369097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.369303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.369575 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] PATH_INFO: `/v3/projects/c60c99ff3e5a4690bcb93b4c08bb2b2b` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.370762 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60c99ff3e5a4690bcb93b4c08bb2b2b)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:28.372742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:28.422854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60c99ff3e5a4690bcb93b4c08bb2b2b', 'request_id': 'req-0fbedfa1-d984-4108-b0a8-66067f2ae787'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.428891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60c99ff3e5a4690bcb93b4c08bb2b2b', 'request_id': 'req-0fbedfa1-d984-4108-b0a8-66067f2ae787'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.438113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60c99ff3e5a4690bcb93b4c08bb2b2b', 'request_id': 'req-0fbedfa1-d984-4108-b0a8-66067f2ae787'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.450334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] The token cache is being invalidate because project c60c99ff3e5a4690bcb93b4c08bb2b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:28.450756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.451255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.452855 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1288/2632] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:28 2022] DELETE /identity/v3/projects/c60c99ff3e5a4690bcb93b4c08bb2b2b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:28.457253 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:28.468795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3790d-c07a-4e35-a62b-a8bb68f4fcd0 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:28.546236 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:28.546662 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.546937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.547202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] PATH_INFO: `/v3/users/f38569b77c9d4bfb82f50ea039f409dd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.548592 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38569b77c9d4bfb82f50ea039f409dd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:28.553739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:28.579293 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:28.611379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38569b77c9d4bfb82f50ea039f409dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.612197 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38569b77c9d4bfb82f50ea039f409dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.621022 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38569b77c9d4bfb82f50ea039f409dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.634547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38569b77c9d4bfb82f50ea039f409dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.641801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-25e95d86-568d-40d8-88e9-a08297d1eb0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38569b77c9d4bfb82f50ea039f409dd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:28.648127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1345/2633] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:28 2022] DELETE /identity/v3/users/f38569b77c9d4bfb82f50ea039f409dd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:28.941557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:28.943150 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.959729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbedfa1-d984-4108-b0a8-66067f2ae787 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.052481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.053192 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.053327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.053604 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] PATH_INFO: `/v3/projects/9b97eea37ba84758a977aaa4dda47ed7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.055372 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b97eea37ba84758a977aaa4dda47ed7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.057343 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.112457 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b97eea37ba84758a977aaa4dda47ed7', 'request_id': 'req-f11aaa17-11cb-4935-845b-0dc6eedca932'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:29.119300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b97eea37ba84758a977aaa4dda47ed7', 'request_id': 'req-f11aaa17-11cb-4935-845b-0dc6eedca932'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:29.129264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b97eea37ba84758a977aaa4dda47ed7', 'request_id': 'req-f11aaa17-11cb-4935-845b-0dc6eedca932'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:29.142567 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] The token cache is being invalidate because project 9b97eea37ba84758a977aaa4dda47ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:29.143106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:29.143950 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f11aaa17-11cb-4935-845b-0dc6eedca932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:29.145838 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1289/2634] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:28 2022] DELETE /identity/v3/projects/9b97eea37ba84758a977aaa4dda47ed7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:29.151298 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.151607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.152037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.175055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:29.176686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:29.244035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1346/2635] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:29.253118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.253318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.253619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.269569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:29.276499 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:29.278736 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:29.288351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1290/2636] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:29.293126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.304008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad74e-1499-4f67-ae7b-cc7bdab743e4 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.310103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.310103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.310103 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.312794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.312794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:29.312794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.313098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.327161 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1347/2637] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:29.332274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.343311 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24023bc-bc22-48b2-bf6c-1db46ebb6102 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.349538 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.350181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.350452 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.350816 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.351511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.353559 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.386134 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1291/2638] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:29.390770 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.402055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3fd5e-f875-4f86-af61-c6cf203d7f49 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.465534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.466167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.466442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.466709 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.467363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.469316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.499665 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1348/2639] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:29.501161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.513095 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7685ff-c882-43b9-a1c8-6ba96f301100 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.518647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.519704 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.519974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.520249 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.521057 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.522969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.534593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1292/2640] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:29.539457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.550422 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca3311-dcd1-4fed-acd0-8ed86d33a0f2 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.559128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ba16b-db05-4d4d-91a3-720bbf6f2d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.559676 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba16b-db05-4d4d-91a3-720bbf6f2d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.559989 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba16b-db05-4d4d-91a3-720bbf6f2d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.560424 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba16b-db05-4d4d-91a3-720bbf6f2d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.561162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ba16b-db05-4d4d-91a3-720bbf6f2d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.563095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ba16b-db05-4d4d-91a3-720bbf6f2d42 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.576899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1349/2641] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:29.581688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.593847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc883f7c-b8b2-49af-a75d-6f34c908a973 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.600934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.601403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.601676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.601936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] PATH_INFO: `/v3/projects/eb98e1b6f2aa42f5bcc24ea409d8ba7a/users/cb94ec2fcade4ee88a484eae840269da/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.620118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb98e1b6f2aa42f5bcc24ea409d8ba7a, user_id=cb94ec2fcade4ee88a484eae840269da, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.622835 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.641562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1293/2642] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:29 2022] PUT /identity/v3/projects/eb98e1b6f2aa42f5bcc24ea409d8ba7a/users/cb94ec2fcade4ee88a484eae840269da/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:29.652568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.652826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.653130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.661699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:29.687596 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] MFA Rules not processed for user `cb94ec2fcade4ee88a484eae840269da`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:29.688860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:29.756823 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1350/2643] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:29.761647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.827573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da86e0-e701-4e9b-857e-57970c1677b4 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.833005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.833743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.833910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.834152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.834852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.837303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.866288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1294/2644] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:29.870564 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.882964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed40e3d-d445-4810-a39a-4be5d8bfd553 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.945679 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.946307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.946589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.946839 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.947544 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:29.949518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:29.976906 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1351/2645] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:29 2022] POST /identity/v3/users => generated 306 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:29.981168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:29.992240 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b353c47-442f-4b3e-a321-309434cbde89 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:29.997335 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:29.997958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:29.998237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:29.998509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:29.999193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:30.000859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:30.013752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1295/2646] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:30.018641 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:30.029729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7e813-3c0a-44d3-a27e-a0ec38a04c3a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:30.035092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:30.035866 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:30.036148 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:30.036419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] PATH_INFO: `/v3/projects/27fbe4f4afa54b43b71db15f09e53318/users/c840876e22754849881ae65389623bfd/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:30.053628 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fbe4f4afa54b43b71db15f09e53318, user_id=c840876e22754849881ae65389623bfd, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:30.056436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:30.074552 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1352/2647] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:30 2022] PUT /identity/v3/projects/27fbe4f4afa54b43b71db15f09e53318/users/c840876e22754849881ae65389623bfd/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:30.078976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:30.089667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d093b-f1d0-428d-9f93-94db326dff5e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:30.150686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6dc04-1e3a-4b05-9571-a084e39ad64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:30.151667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6dc04-1e3a-4b05-9571-a084e39ad64e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:30.151982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6dc04-1e3a-4b05-9571-a084e39ad64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:30.152256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6dc04-1e3a-4b05-9571-a084e39ad64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:30.152958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6dc04-1e3a-4b05-9571-a084e39ad64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:30.154576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6dc04-1e3a-4b05-9571-a084e39ad64e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:30.165381 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1296/2648] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:30.169851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:30.180925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bcc2c-28f0-4210-abb8-a368348d87cc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:30.186720 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:30.187378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:30.187654 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:30.187953 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] PATH_INFO: `/v3/projects/27fbe4f4afa54b43b71db15f09e53318/users/c840876e22754849881ae65389623bfd/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:30.190146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fbe4f4afa54b43b71db15f09e53318, user_id=c840876e22754849881ae65389623bfd, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:30.193085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:30.205091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1353/2649] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:30 2022] PUT /identity/v3/projects/27fbe4f4afa54b43b71db15f09e53318/users/c840876e22754849881ae65389623bfd/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:30.216929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:30.217233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:30.217505 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:30.247172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:30.250576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] MFA Rules not processed for user `c840876e22754849881ae65389623bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:30.252781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:30.329120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1297/2650] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:30.340985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:30.343453 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:30.399752 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:30.412712 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d0a0-cc9f-4741-9385-4df9cda1dd72 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:30.416946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:30.417597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:30.417864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:30.418158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:30.430680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:30.432816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:30.446086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1354/2651] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:30.532636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:30.543089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f658d67-166f-41b0-b0c6-70d527e8cc79 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:30.547238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897b22-8307-427c-b001-4e73c6a289e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:30.548094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897b22-8307-427c-b001-4e73c6a289e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:30.548437 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897b22-8307-427c-b001-4e73c6a289e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:30.548865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897b22-8307-427c-b001-4e73c6a289e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:30.612966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897b22-8307-427c-b001-4e73c6a289e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:30.614340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897b22-8307-427c-b001-4e73c6a289e3 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:30.624592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1298/2652] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:33.600234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:33.602152 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:33.673675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3068f-25e9-4003-b070-60d94c1e3360 service cinder] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:33.681249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:33.681598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:33.681934 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:33.682268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] PATH_INFO: `/v3/users/75feb70b6a51485aa62290467603a71c` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:33.683891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=75feb70b6a51485aa62290467603a71c)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:33.686274 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:33.743865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75feb70b6a51485aa62290467603a71c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:33.744531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75feb70b6a51485aa62290467603a71c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:33.757565 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75feb70b6a51485aa62290467603a71c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:33.772624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75feb70b6a51485aa62290467603a71c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:33.782331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75feb70b6a51485aa62290467603a71c'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:33.791170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1355/2653] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:33 2022] DELETE /identity/v3/users/75feb70b6a51485aa62290467603a71c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:33.796203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:33.796800 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:33.796979 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:33.821081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:33.822780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:33.929146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1299/2654] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:33.942344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:33.946394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:34.024587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739f72-89bb-4df1-8c54-1c156485aaea admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:34.031929 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:34.036124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:34.037438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:34.039776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:34.057904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:34.060171 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:34.073874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1356/2655] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:34.414757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:34.416780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:34.429892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3506fac-c15d-47ec-82de-b1133bdeefb0 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:34.435195 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:34.436052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:34.436276 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:34.436549 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] PATH_INFO: `/v3/projects/5c995a4d9d2a4778b2d75214f1b35518` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:34.437774 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c995a4d9d2a4778b2d75214f1b35518)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:34.439793 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:34.483305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:34.498901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c995a4d9d2a4778b2d75214f1b35518', 'request_id': 'req-03712705-d51f-40b8-8714-06a365f15ed6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:34.505033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c995a4d9d2a4778b2d75214f1b35518', 'request_id': 'req-03712705-d51f-40b8-8714-06a365f15ed6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:34.513991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c995a4d9d2a4778b2d75214f1b35518', 'request_id': 'req-03712705-d51f-40b8-8714-06a365f15ed6'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:34.526147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] The token cache is being invalidate because project 5c995a4d9d2a4778b2d75214f1b35518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:34.526512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:34.527108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-03712705-d51f-40b8-8714-06a365f15ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:34.528755 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1300/2656] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:34 2022] DELETE /identity/v3/projects/5c995a4d9d2a4778b2d75214f1b35518 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:34.924159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:34.926023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:34.938897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be06f4a-ad33-4a21-9889-5effaf3e1b8c service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:34.944141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:35.001389 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:35.001950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:35.002259 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:35.002531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] PATH_INFO: `/v3/users/cb94ec2fcade4ee88a484eae840269da` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:35.003817 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb94ec2fcade4ee88a484eae840269da)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:35.005745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:35.063514 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb94ec2fcade4ee88a484eae840269da'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.064303 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb94ec2fcade4ee88a484eae840269da'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.076270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb94ec2fcade4ee88a484eae840269da'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.091507 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb94ec2fcade4ee88a484eae840269da'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.099430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb94ec2fcade4ee88a484eae840269da'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.105727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1357/2657] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:34 2022] DELETE /identity/v3/users/cb94ec2fcade4ee88a484eae840269da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:35.110396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:35.113454 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:35.113778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:35.133663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:35.134893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:35.202217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1301/2658] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:35.212660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:35.214289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:35.275478 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d4753-3af9-494b-b693-dc3f1f860f84 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:35.279268 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:35.279916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:35.280191 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:35.280458 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:35.290525 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:35.291899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:35.302097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1358/2659] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:35.590303 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:35.592032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:35.608067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f105e-2b2a-42e1-8101-20bff354d9ab None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:35.613963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:35.614791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:35.615072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:35.615347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] PATH_INFO: `/v3/projects/eb98e1b6f2aa42f5bcc24ea409d8ba7a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:35.616693 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb98e1b6f2aa42f5bcc24ea409d8ba7a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:35.618699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:35.671373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb98e1b6f2aa42f5bcc24ea409d8ba7a', 'request_id': 'req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.677892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb98e1b6f2aa42f5bcc24ea409d8ba7a', 'request_id': 'req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.687501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb98e1b6f2aa42f5bcc24ea409d8ba7a', 'request_id': 'req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.699313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] The token cache is being invalidate because project eb98e1b6f2aa42f5bcc24ea409d8ba7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:35.699598 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.700333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.701894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1302/2660] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:35 2022] DELETE /identity/v3/projects/eb98e1b6f2aa42f5bcc24ea409d8ba7a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:35.705978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:35.716690 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06924bec-23f0-4b16-b121-984ef4ca9d5a service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:35.748609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:35.780012 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:35.780655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:35.780924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:35.781189 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] PATH_INFO: `/v3/users/c840876e22754849881ae65389623bfd` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:35.782405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c840876e22754849881ae65389623bfd)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:35.784291 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:35.838627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c840876e22754849881ae65389623bfd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.839403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c840876e22754849881ae65389623bfd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.849068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c840876e22754849881ae65389623bfd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.861882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c840876e22754849881ae65389623bfd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.869700 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a75a4aee-eb34-4a37-abe7-df2c74f9b5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c840876e22754849881ae65389623bfd'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:35.876299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1359/2661] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:35 2022] DELETE /identity/v3/users/c840876e22754849881ae65389623bfd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:36.173207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.174865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:36.190041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221f15f-ba2e-46b6-b8c9-5ecf9446cb98 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.239400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:36.260404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.261072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.261348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.261621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] PATH_INFO: `/v3/projects/27fbe4f4afa54b43b71db15f09e53318` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.263094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27fbe4f4afa54b43b71db15f09e53318)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.265176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.324947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fbe4f4afa54b43b71db15f09e53318', 'request_id': 'req-e7a219eb-1c67-41bf-a46c-8a982d551a41'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:36.333194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fbe4f4afa54b43b71db15f09e53318', 'request_id': 'req-e7a219eb-1c67-41bf-a46c-8a982d551a41'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:36.346213 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fbe4f4afa54b43b71db15f09e53318', 'request_id': 'req-e7a219eb-1c67-41bf-a46c-8a982d551a41'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:36.365928 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] The token cache is being invalidate because project 27fbe4f4afa54b43b71db15f09e53318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:36.366340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:36.366881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-e7a219eb-1c67-41bf-a46c-8a982d551a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:36.368695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1303/2662] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:36 2022] DELETE /identity/v3/projects/27fbe4f4afa54b43b71db15f09e53318 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:36.373703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.374200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.374300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.396338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:36.397890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:36.463479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1360/2663] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:36.471483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.471746 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.472228 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.494672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:36.496064 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:36.505488 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1304/2664] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:36.510182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.520023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986243-84dd-4f16-8f71-2cc56325f97c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.524887 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.525513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.525780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.526047 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.526746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:36.527044 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.528687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.541457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1361/2665] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:36.546020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.558104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2a63-0e55-44a6-8734-d9908e3c6e7d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.564668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.565298 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.565690 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.565930 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.566789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.568756 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.600839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1305/2666] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:36 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:36.605212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.616302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756bbd0-9f2d-40d6-8336-03e33e9e284f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.656038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:36.690950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.691584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.691892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.692162 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.692869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.694818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.717240 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1362/2667] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:36 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:46:36.721742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.733282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01156b66-57d5-4159-aa85-3f9a336dba1c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.738483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.739104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.739382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.739669 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.740433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.742041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.754376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1306/2668] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:36.757878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.769244 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796cd64-7487-4cfd-934c-93efd988344f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.774789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41004c6e-865e-49b5-b691-68af10ac17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.775413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41004c6e-865e-49b5-b691-68af10ac17f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.775688 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41004c6e-865e-49b5-b691-68af10ac17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.775995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41004c6e-865e-49b5-b691-68af10ac17f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.776699 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41004c6e-865e-49b5-b691-68af10ac17f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.778312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41004c6e-865e-49b5-b691-68af10ac17f0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.789093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1363/2669] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:36.793732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:36.804441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9f129-81d6-4259-921e-3b4bb608e05a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:36.809724 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:36.810397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.810674 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.810944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] PATH_INFO: `/v3/projects/23ec8720205a4943af3b2a89cbc526d5/users/f021ad1116e3490f8425dbbbc8f78ef1/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.827241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ec8720205a4943af3b2a89cbc526d5, user_id=f021ad1116e3490f8425dbbbc8f78ef1, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:36.829963 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:36.845340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1307/2670] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:36 2022] PUT /identity/v3/projects/23ec8720205a4943af3b2a89cbc526d5/users/f021ad1116e3490f8425dbbbc8f78ef1/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:36.857436 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:36.857613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:36.857948 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:36.887329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] MFA Rules not processed for user `f021ad1116e3490f8425dbbbc8f78ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:36.888624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:36.956736 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1364/2671] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:36.961987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.033634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e50e3c-c66b-4515-8c64-883938617154 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.039379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.040040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.040318 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.040588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.041256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.043187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.059473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:37.075871 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1308/2672] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:36 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:37.079708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.091870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fab39-6893-448c-ba64-ced90d487731 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.158971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.159615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.159926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.160213 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.160874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.162821 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.186786 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1365/2673] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:37 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:37.191633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.202886 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2d39d-5703-4630-850c-c1851a4dcfdd admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.208113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.208741 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.209016 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.209281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.209981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.211929 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.223551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1309/2674] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:37.227924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.238536 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68becb32-36fa-43c5-8ce5-b15ca90f7e6f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.243575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.244275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.244578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.244791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] PATH_INFO: `/v3/projects/1ea2472d1cf34f25bec832747e949594/users/c3db15f171dd4b6c9ebbec6c362a9219/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.260942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea2472d1cf34f25bec832747e949594, user_id=c3db15f171dd4b6c9ebbec6c362a9219, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.263616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.278861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1366/2675] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:37 2022] PUT /identity/v3/projects/1ea2472d1cf34f25bec832747e949594/users/c3db15f171dd4b6c9ebbec6c362a9219/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:37.283035 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.295020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb655e78-a253-4645-b16c-7fa808458925 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.357041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba0c21-7e18-4cdc-bed1-7c8303b8b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.357666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba0c21-7e18-4cdc-bed1-7c8303b8b805 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.357944 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba0c21-7e18-4cdc-bed1-7c8303b8b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.358209 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba0c21-7e18-4cdc-bed1-7c8303b8b805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.358912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba0c21-7e18-4cdc-bed1-7c8303b8b805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.360562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba0c21-7e18-4cdc-bed1-7c8303b8b805 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.371633 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1310/2676] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:37.376060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.387575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08953e-bb6e-4b2a-a956-eb27afba5c0c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.393460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.394350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.394692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.395024 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] PATH_INFO: `/v3/projects/1ea2472d1cf34f25bec832747e949594/users/c3db15f171dd4b6c9ebbec6c362a9219/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.397648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea2472d1cf34f25bec832747e949594, user_id=c3db15f171dd4b6c9ebbec6c362a9219, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.400405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.408131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:37.415002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1367/2677] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:37 2022] PUT /identity/v3/projects/1ea2472d1cf34f25bec832747e949594/users/c3db15f171dd4b6c9ebbec6c362a9219/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:37.491023 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.491689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.492137 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.538002 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] MFA Rules not processed for user `c3db15f171dd4b6c9ebbec6c362a9219`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:37.538770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:37.620451 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1311/2678] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:37.633069 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.634868 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:37.698692 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b48a9f-6bcb-48af-a38f-dba17fdafd09 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.702351 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f40e34-6963-41d8-b8a4-a1ac889bb8ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.702971 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f40e34-6963-41d8-b8a4-a1ac889bb8ed service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.703246 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f40e34-6963-41d8-b8a4-a1ac889bb8ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.703518 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f40e34-6963-41d8-b8a4-a1ac889bb8ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.717964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f40e34-6963-41d8-b8a4-a1ac889bb8ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.719421 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f40e34-6963-41d8-b8a4-a1ac889bb8ed service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.730068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1368/2679] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:46:37.877188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.943509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2430-2010-462a-a006-cf2c95a41570 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.948260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.948922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.949244 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.949526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] PATH_INFO: `/v3/users/f021ad1116e3490f8425dbbbc8f78ef1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.950770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f021ad1116e3490f8425dbbbc8f78ef1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:37.952757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:37.958940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=100 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:38.006008 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f021ad1116e3490f8425dbbbc8f78ef1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.006666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f021ad1116e3490f8425dbbbc8f78ef1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.018971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f021ad1116e3490f8425dbbbc8f78ef1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.026482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f021ad1116e3490f8425dbbbc8f78ef1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.027220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f021ad1116e3490f8425dbbbc8f78ef1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.043065 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1312/2680] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:37 2022] DELETE /identity/v3/users/f021ad1116e3490f8425dbbbc8f78ef1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:38.051163 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:38.051543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:38.051847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:38.072895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:38.074142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:38.170975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1369/2681] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:38.187849 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:38.189552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:38.256140 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e83ae-ae2a-42e2-94e9-5292e90cfa35 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:38.260102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:38.260823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:38.261124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:38.261415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:38.274047 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:38.275508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:38.287050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1313/2682] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:38.614327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:38.619804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:38.636807 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11c0f-b556-4dd6-9baa-fe577fc6011b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:38.651746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:38.652878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:38.653391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:38.653876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] PATH_INFO: `/v3/projects/23ec8720205a4943af3b2a89cbc526d5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:38.655579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ec8720205a4943af3b2a89cbc526d5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:38.658695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:38.751195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ec8720205a4943af3b2a89cbc526d5', 'request_id': 'req-968a08df-94a0-443e-9c19-6ef2a6c9250a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.759260 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ec8720205a4943af3b2a89cbc526d5', 'request_id': 'req-968a08df-94a0-443e-9c19-6ef2a6c9250a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.770209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ec8720205a4943af3b2a89cbc526d5', 'request_id': 'req-968a08df-94a0-443e-9c19-6ef2a6c9250a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.785109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] The token cache is being invalidate because project 23ec8720205a4943af3b2a89cbc526d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:38.785561 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.786414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.788234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1370/2683] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:38 2022] DELETE /identity/v3/projects/23ec8720205a4943af3b2a89cbc526d5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:38.792973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:38.805550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0da7-f6ea-43af-b2c6-355cac115592 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:38.903485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:38.904376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:38.904786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:38.905173 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] PATH_INFO: `/v3/users/c3db15f171dd4b6c9ebbec6c362a9219` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:38.906490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3db15f171dd4b6c9ebbec6c362a9219)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:38.908616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:38.948777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:38.966571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3db15f171dd4b6c9ebbec6c362a9219'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.967351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3db15f171dd4b6c9ebbec6c362a9219'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.983964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3db15f171dd4b6c9ebbec6c362a9219'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.991560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3db15f171dd4b6c9ebbec6c362a9219'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:38.992403 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-269163ee-f203-41c1-b309-35fcb084871b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3db15f171dd4b6c9ebbec6c362a9219'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:39.007727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1314/2684] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:38 2022] DELETE /identity/v3/users/c3db15f171dd4b6c9ebbec6c362a9219 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:39.310196 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.311675 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:39.326329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a08df-94a0-443e-9c19-6ef2a6c9250a admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:39.350335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:39.389794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:39.390451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.390726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.390992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] PATH_INFO: `/v3/projects/1ea2472d1cf34f25bec832747e949594` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.392195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea2472d1cf34f25bec832747e949594)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:39.394107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:39.447076 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea2472d1cf34f25bec832747e949594', 'request_id': 'req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:39.453296 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea2472d1cf34f25bec832747e949594', 'request_id': 'req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:39.462910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea2472d1cf34f25bec832747e949594', 'request_id': 'req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:39.476884 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] The token cache is being invalidate because project 1ea2472d1cf34f25bec832747e949594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:39.477175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:39.477916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-81042c9f-fb23-49b0-8cf9-ba070fd17ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:39.479618 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1371/2685] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:39 2022] DELETE /identity/v3/projects/1ea2472d1cf34f25bec832747e949594 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:39.485098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.485829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.485829 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.510611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:39.511901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:39.584474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1315/2686] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:39.593135 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.593442 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.593695 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.619526 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:39.620892 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:39.629601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1372/2687] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:39.634242 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.645501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e154798-acc5-49e1-9bab-e4b3ef491274 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:39.652385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:39.653030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.653306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.653578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.654274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:39.654578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:39.656251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:39.673967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1316/2688] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:39.680279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.690093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4806b3-cefd-4d56-9c02-5fff3a309067 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:39.695889 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:39.696589 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.696952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.697138 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.697816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:39.699792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:39.741167 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1373/2689] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:39 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:39.742920 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.756128 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883c9b-4931-4c1f-818b-279d806f10dc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:39.837772 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:39.839282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.840599 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.841555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.842476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:39.846685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:39.876961 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1317/2690] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:39 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:39.881320 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.893778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598c2fa-189e-45aa-a890-3e1af0e12ac7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:39.902088 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:39.902744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.903021 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.903282 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.904091 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:39.905735 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:39.917416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1374/2691] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:39.930720 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.941982 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57f366-bcfe-4c3b-a42c-47198daf5089 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:39.947040 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2fc55-15bd-46a8-97fa-16e580a1bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:39.947678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2fc55-15bd-46a8-97fa-16e580a1bae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:39.948022 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2fc55-15bd-46a8-97fa-16e580a1bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:39.948278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2fc55-15bd-46a8-97fa-16e580a1bae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:39.949001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2fc55-15bd-46a8-97fa-16e580a1bae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:39.950731 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2fc55-15bd-46a8-97fa-16e580a1bae6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:39.965074 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1318/2692] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:39.970680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:39.990165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce54d6-1960-4f9c-90f8-3419bd2103a3 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.004586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.005626 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.005904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.006186 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] PATH_INFO: `/v3/projects/546f427b298a4022b4eb3567a5f8cd42/users/97d4dd1054004186b0cf550baf808aef/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.025582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=546f427b298a4022b4eb3567a5f8cd42, user_id=97d4dd1054004186b0cf550baf808aef, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.029860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.051974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1375/2693] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:39 2022] PUT /identity/v3/projects/546f427b298a4022b4eb3567a5f8cd42/users/97d4dd1054004186b0cf550baf808aef/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:40.064700 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.065241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.065563 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.094172 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:40.098052 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] MFA Rules not processed for user `97d4dd1054004186b0cf550baf808aef`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:40.100183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:40.177306 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1319/2694] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:40.182210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.237572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:40.249623 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d02a8-1deb-4cd6-9b3c-5fbc39a484ba admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.255938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.256581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.256816 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.257080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.257743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.259647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.289513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1376/2695] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:40 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:40.294543 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.307251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d31d81-5b27-4ff6-8467-7eef0de64ca4 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.370713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.371328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.371603 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.371901 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.372572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.374556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.399844 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1320/2696] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:40 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:40.402051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.413583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19adc923-09ed-459f-afbe-820ad21cce9b admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.418812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.419430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.419703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.419998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.420959 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.422297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.437119 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1377/2697] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:40.438433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.449952 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41781c6a-1444-4ebd-8d8f-d313d2d07dc0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.455353 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.456180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.456576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.456757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] PATH_INFO: `/v3/projects/67ff0b76a5c54452ab948fc028999d22/users/c3e6ca19142740e6b6dc81c6cb0f7f96/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.474348 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ff0b76a5c54452ab948fc028999d22, user_id=c3e6ca19142740e6b6dc81c6cb0f7f96, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.477332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.494853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1321/2698] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:40 2022] PUT /identity/v3/projects/67ff0b76a5c54452ab948fc028999d22/users/c3e6ca19142740e6b6dc81c6cb0f7f96/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:40.499300 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.510257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb385d-e2fb-4706-87b2-258a6bd966bb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.569691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9eca1-6b1b-49a1-a79c-52a9759552ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.570499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9eca1-6b1b-49a1-a79c-52a9759552ea admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.570769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9eca1-6b1b-49a1-a79c-52a9759552ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.571035 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9eca1-6b1b-49a1-a79c-52a9759552ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.571722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9eca1-6b1b-49a1-a79c-52a9759552ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.573998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9eca1-6b1b-49a1-a79c-52a9759552ea admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.585039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1378/2699] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:40.589934 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.601218 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097e4f-94a3-4ef6-93d6-022f818270af admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.606522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.607116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.607384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.607648 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] PATH_INFO: `/v3/projects/67ff0b76a5c54452ab948fc028999d22/users/c3e6ca19142740e6b6dc81c6cb0f7f96/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.609689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ff0b76a5c54452ab948fc028999d22, user_id=c3e6ca19142740e6b6dc81c6cb0f7f96, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.612339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.623713 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1322/2700] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:40 2022] PUT /identity/v3/projects/67ff0b76a5c54452ab948fc028999d22/users/c3e6ca19142740e6b6dc81c6cb0f7f96/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:40.634473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.635156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.635156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.665019 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] MFA Rules not processed for user `c3e6ca19142740e6b6dc81c6cb0f7f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:40.666132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:40.742011 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1379/2701] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:40.754050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:40.756320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:40.831433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712571e-f0f0-4604-9ddb-74623fa0cebb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:40.835820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4928c-ef6e-4af2-b3c1-687cb0548098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:40.836508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4928c-ef6e-4af2-b3c1-687cb0548098 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:40.836777 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4928c-ef6e-4af2-b3c1-687cb0548098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:40.837042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4928c-ef6e-4af2-b3c1-687cb0548098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:40.846798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4928c-ef6e-4af2-b3c1-687cb0548098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:40.848642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4928c-ef6e-4af2-b3c1-687cb0548098 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:40.852917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:40.860865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1323/2702] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:41.327636 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:41.398128 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:41.418228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74a55f-e873-4168-969f-cd84ec21eb41 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:41.423064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:41.423922 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:41.424487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:41.425350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] PATH_INFO: `/v3/users/97d4dd1054004186b0cf550baf808aef` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:41.426249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d4dd1054004186b0cf550baf808aef)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:41.428403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:41.491356 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d4dd1054004186b0cf550baf808aef'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:41.492386 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d4dd1054004186b0cf550baf808aef'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:41.505946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d4dd1054004186b0cf550baf808aef'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:41.524726 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d4dd1054004186b0cf550baf808aef'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:41.537614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d4dd1054004186b0cf550baf808aef'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:41.544615 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1380/2703] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:41 2022] DELETE /identity/v3/users/97d4dd1054004186b0cf550baf808aef => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:41.549927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:41.549927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:41.550583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:41.574370 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:41.575638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:41.655406 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1324/2704] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:41.673858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:41.673858 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:41.757297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498e64-c862-4dd5-8fbc-745bcc46181f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:41.762349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:41.763382 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:41.763870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:41.764479 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:41.775038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:41.776541 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:41.787725 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1381/2705] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:42.096779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:42.097830 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:42.117524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ef0fe-baa0-4d25-905e-829b67e41bd2 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:42.148665 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:42.149255 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:42.149540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:42.149813 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] PATH_INFO: `/v3/projects/546f427b298a4022b4eb3567a5f8cd42` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:42.151238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] RBAC: Authorizing `identity:delete_project(project_id=546f427b298a4022b4eb3567a5f8cd42)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:42.153727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:42.224181 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546f427b298a4022b4eb3567a5f8cd42', 'request_id': 'req-dcf01f17-800b-4699-9942-04677086105c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.230723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546f427b298a4022b4eb3567a5f8cd42', 'request_id': 'req-dcf01f17-800b-4699-9942-04677086105c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.241432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546f427b298a4022b4eb3567a5f8cd42', 'request_id': 'req-dcf01f17-800b-4699-9942-04677086105c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.254462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] The token cache is being invalidate because project 546f427b298a4022b4eb3567a5f8cd42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:42.254854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.255584 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.257223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1325/2706] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:42 2022] DELETE /identity/v3/projects/546f427b298a4022b4eb3567a5f8cd42 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:42.261388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:42.272265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78333015-0351-406a-8785-4097558d1f31 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:42.349216 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:42.349910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:42.350192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:42.350466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] PATH_INFO: `/v3/users/c3e6ca19142740e6b6dc81c6cb0f7f96` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:42.352219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e6ca19142740e6b6dc81c6cb0f7f96)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:42.354396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:42.367911 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:42.422936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e6ca19142740e6b6dc81c6cb0f7f96'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.423803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e6ca19142740e6b6dc81c6cb0f7f96'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.433477 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e6ca19142740e6b6dc81c6cb0f7f96'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.453843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e6ca19142740e6b6dc81c6cb0f7f96'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.463504 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d1e9b067-3716-4fae-8316-693c4a2899ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e6ca19142740e6b6dc81c6cb0f7f96'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.470904 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1382/2707] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:42 2022] DELETE /identity/v3/users/c3e6ca19142740e6b6dc81c6cb0f7f96 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:42.774814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:42.776345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:42.789850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf01f17-800b-4699-9942-04677086105c admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:42.850627 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:42.851273 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:42.851547 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:42.851865 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] PATH_INFO: `/v3/projects/67ff0b76a5c54452ab948fc028999d22` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:42.852983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ff0b76a5c54452ab948fc028999d22)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:42.854896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:42.880535 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:42.911478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ff0b76a5c54452ab948fc028999d22', 'request_id': 'req-d69d544d-bb81-4d38-be29-a51c3fe76148'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.917881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ff0b76a5c54452ab948fc028999d22', 'request_id': 'req-d69d544d-bb81-4d38-be29-a51c3fe76148'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.927479 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ff0b76a5c54452ab948fc028999d22', 'request_id': 'req-d69d544d-bb81-4d38-be29-a51c3fe76148'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.941968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] The token cache is being invalidate because project 67ff0b76a5c54452ab948fc028999d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:42.942597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.943374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d69d544d-bb81-4d38-be29-a51c3fe76148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:42.945049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1326/2708] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:42 2022] DELETE /identity/v3/projects/67ff0b76a5c54452ab948fc028999d22 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:42.950105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:42.950658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:42.950658 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:42.973136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:42.974404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.037159 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1383/2709] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:43.046193 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297981-faef-4f89-bb87-1fb940be62df None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.046392 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297981-faef-4f89-bb87-1fb940be62df None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.046641 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297981-faef-4f89-bb87-1fb940be62df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.070320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-95297981-faef-4f89-bb87-1fb940be62df None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:43.071573 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-95297981-faef-4f89-bb87-1fb940be62df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.079959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1327/2710] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:43.085548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.099141 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea97f-a922-4b5e-b492-c2375b6b302f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.105642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.106330 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.106612 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.106900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.107617 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:43.107946 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.109593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.124519 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1384/2711] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:43.130478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297981-faef-4f89-bb87-1fb940be62df None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.140000 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297981-faef-4f89-bb87-1fb940be62df None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.144971 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.145702 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.145904 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.146166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.146828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.148789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.179418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1328/2712] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:43 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:43.184693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.196986 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dea167-eaef-410c-b40c-d743de1a026d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.262606 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.263233 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.263508 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.263893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.264464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.266403 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.280284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.282186 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.295431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b374c8-ede7-46f1-b606-97ae45f66033 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.299826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1385/2713] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:43 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:43.300265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.301018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.301285 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.301550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] PATH_INFO: `/v3/users/03880809413c43629852d3f9dc865d32` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.302853 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] RBAC: Authorizing `identity:delete_user(user_id=03880809413c43629852d3f9dc865d32)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.304396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.305018 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.318683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:43.323114 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26472a-2a5d-4718-bf30-9c0f1e07aba7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.331995 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.333375 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.333502 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.333739 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.334802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.336742 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.349846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1386/2714] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:43.354264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.364729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534855b-b336-4508-b4db-bdbf9da058a5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.369951 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.370590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.370865 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.371127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.371870 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.373535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.374712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03880809413c43629852d3f9dc865d32'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:43.383639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03880809413c43629852d3f9dc865d32'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:43.388002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1387/2715] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:43.392094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.403961 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f5a8-b3c6-421a-a4c1-a0fe1a6c11bc admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.408493 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03880809413c43629852d3f9dc865d32'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:43.409633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7d33f-25d2-4f0e-8770-bbb443db2689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.410520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7d33f-25d2-4f0e-8770-bbb443db2689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.411165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7d33f-25d2-4f0e-8770-bbb443db2689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.411165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7d33f-25d2-4f0e-8770-bbb443db2689 admin admin] PATH_INFO: `/v3/projects/7599805e90374ddbbd17bf9899bdc8b2/users/52ad0bd481a846f6b5c807fd69904201/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.420815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03880809413c43629852d3f9dc865d32'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:43.421831 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-194cac35-11c6-4ffd-82c6-c21affd5825b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03880809413c43629852d3f9dc865d32'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:43.427642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7d33f-25d2-4f0e-8770-bbb443db2689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7599805e90374ddbbd17bf9899bdc8b2, user_id=52ad0bd481a846f6b5c807fd69904201, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.430579 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7d33f-25d2-4f0e-8770-bbb443db2689 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.440302 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1329/2716] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:43 2022] DELETE /identity/v3/users/03880809413c43629852d3f9dc865d32 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:43.445551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.445906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.446248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.446680 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1388/2717] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:43 2022] PUT /identity/v3/projects/7599805e90374ddbbd17bf9899bdc8b2/users/52ad0bd481a846f6b5c807fd69904201/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:43.458616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.458804 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.459057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.468334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:43.469518 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.489859 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] MFA Rules not processed for user `52ad0bd481a846f6b5c807fd69904201`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:43.491210 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.524085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:43.541207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1330/2718] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:43.551355 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.552826 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.570226 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1389/2719] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:43.582086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.583677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.613411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153efe-1ec3-4931-893a-1de73e56034c None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.617804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.618439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.618711 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.618976 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.628919 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.630933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.641490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1331/2720] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:43.643243 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d015fc-e58e-4b15-adc5-32aad57ee31b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.647192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741a12c-47dc-44d0-ad0d-79fb1e465ee0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288250b6b648409aa179b3213321b869', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.647851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a12c-47dc-44d0-ad0d-79fb1e465ee0 service glance] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.648165 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a12c-47dc-44d0-ad0d-79fb1e465ee0 service glance] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.648397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a12c-47dc-44d0-ad0d-79fb1e465ee0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.658039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a12c-47dc-44d0-ad0d-79fb1e465ee0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.659689 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a12c-47dc-44d0-ad0d-79fb1e465ee0 service glance] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:43.670379 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1390/2721] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:43.942221 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:43.943739 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:43.957136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24955257-5bbc-4476-bbb2-6b27454ebc5d service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:43.963884 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:43.964582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.964852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.965127 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] PATH_INFO: `/v3/projects/b0e048a61c324891a03130fff8bda0b1` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.966385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e048a61c324891a03130fff8bda0b1)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:43.968423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.024066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e048a61c324891a03130fff8bda0b1', 'request_id': 'req-d459c73d-59eb-4f32-bcf8-f0f520929b49'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:44.031566 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e048a61c324891a03130fff8bda0b1', 'request_id': 'req-d459c73d-59eb-4f32-bcf8-f0f520929b49'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:44.041268 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e048a61c324891a03130fff8bda0b1', 'request_id': 'req-d459c73d-59eb-4f32-bcf8-f0f520929b49'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:44.054522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] The token cache is being invalidate because project b0e048a61c324891a03130fff8bda0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:44.054809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:44.055471 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-d459c73d-59eb-4f32-bcf8-f0f520929b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:44.057362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1332/2722] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:43 2022] DELETE /identity/v3/projects/b0e048a61c324891a03130fff8bda0b1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:44.062254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15668898-8894-4691-bf05-8054fc689c17 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.062806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15668898-8894-4691-bf05-8054fc689c17 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.062806 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15668898-8894-4691-bf05-8054fc689c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.083754 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-15668898-8894-4691-bf05-8054fc689c17 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:44.085451 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-15668898-8894-4691-bf05-8054fc689c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.148833 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:44.167744 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1391/2723] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:44.181666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.181980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.182157 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.202714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:44.204070 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.213246 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1333/2724] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:44.217635 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15668898-8894-4691-bf05-8054fc689c17 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.228910 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15668898-8894-4691-bf05-8054fc689c17 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.235249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.236008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.236276 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.236563 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.237270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:44.237571 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.239359 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.251733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1392/2725] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:44.256613 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.266300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e957c6-f9ad-47dc-84ab-b0adeca6a116 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.272676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.273580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.273824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.274092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.274797 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.276783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.311108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1334/2726] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:44 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:44.312489 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.323799 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea84a6-505e-4fa6-8c8c-2ec3583a80d5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.386994 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.387631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.387941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.388214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.388879 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.390820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.421657 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1393/2727] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:44 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:44.422866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.438895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be731919-66bf-410c-9a1f-825ad76594da admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.444691 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.445378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.445661 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.445933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.446659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.448316 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.459090 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1335/2728] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:44.463624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.478674 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c3430-7bed-4cad-80c5-2f29c8465aec admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.486509 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530bc05-ca03-4a76-894c-cc80629d0dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.487443 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530bc05-ca03-4a76-894c-cc80629d0dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.487697 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530bc05-ca03-4a76-894c-cc80629d0dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.488064 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530bc05-ca03-4a76-894c-cc80629d0dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.489332 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530bc05-ca03-4a76-894c-cc80629d0dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.492092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530bc05-ca03-4a76-894c-cc80629d0dcf admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.510797 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1394/2729] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:44.515962 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.527683 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae39001-9aba-49ab-9193-e18b93644297 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.533955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.534770 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.535142 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.535419 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] PATH_INFO: `/v3/projects/01e1704c3b1e4729820088ca9ed48368/users/39593dbdff134fb1af13b341d6c01e37/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.551671 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e1704c3b1e4729820088ca9ed48368, user_id=39593dbdff134fb1af13b341d6c01e37, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.554420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.570749 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1336/2730] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:44 2022] PUT /identity/v3/projects/01e1704c3b1e4729820088ca9ed48368/users/39593dbdff134fb1af13b341d6c01e37/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:44.576819 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d967f-f747-473f-a3ce-cf9fe7e84e7a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.577108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d967f-f747-473f-a3ce-cf9fe7e84e7a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.577378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d967f-f747-473f-a3ce-cf9fe7e84e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.601342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-de8d967f-f747-473f-a3ce-cf9fe7e84e7a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:44.602637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-de8d967f-f747-473f-a3ce-cf9fe7e84e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.667124 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1395/2731] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:44.677866 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.679728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.701423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:44.741232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b321282-c9fd-4073-a7f7-da4ed91a5d7a admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.745395 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.746030 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.746430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.746706 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.759380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.763153 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.785257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1337/2732] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:45.478363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07f986-c469-4206-823f-cf6b242e00aa None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:45.478938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07f986-c469-4206-823f-cf6b242e00aa None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:45.478938 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07f986-c469-4206-823f-cf6b242e00aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:45.514466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-ff07f986-c469-4206-823f-cf6b242e00aa None None] MFA Rules not processed for user `39593dbdff134fb1af13b341d6c01e37`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:45.515722 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ff07f986-c469-4206-823f-cf6b242e00aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:45.569593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:45.593030 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1396/2733] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:45.604642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:45.607178 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:45.686337 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461e48-6fb2-4d23-b509-4400e972b734 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:45.690093 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:45.690723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:45.691001 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:45.691328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:45.701266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:45.702719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:45.713376 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1338/2734] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:47.097583 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:47.098127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:47.098127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:47.117677 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] MFA Rules not processed for user `39593dbdff134fb1af13b341d6c01e37`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:47.118877 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:47.126365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1397/2735] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:53.861857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:53.886685 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6e48f-e1aa-41a3-b365-21028f163a9e service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:53.894877 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:53.896180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:53.896467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:53.896740 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:53.898714 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:53.967796 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:53.969483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:53.992859 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1339/2736] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:57.112050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=9 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:57.752002 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:57.754316 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:57.861716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:57.864435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:57.880990 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10273-fc49-41d4-84cb-984194b4e143 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:57.887344 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:57.888187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:57.888456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:57.888723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] PATH_INFO: `/v3/users/76c1fab92b854cd8937cc80818159191` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:57.889980 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c1fab92b854cd8937cc80818159191)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:57.892166 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:57.949427 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:57.963201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c1fab92b854cd8937cc80818159191'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:57.963870 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c1fab92b854cd8937cc80818159191'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:57.977841 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c1fab92b854cd8937cc80818159191'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:57.987512 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c1fab92b854cd8937cc80818159191'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:57.988606 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c1fab92b854cd8937cc80818159191'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:58.009805 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2178a5-05d0-4ca5-94f4-a6a1ca45223c None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.016310 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1340/2737] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:46:57 2022] DELETE /identity/v3/users/76c1fab92b854cd8937cc80818159191 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:58.016663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.016663 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.016875 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.017647 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.019007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:58.033083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.034503 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e1a6c-f8c6-4e54-a193-0ef4036daf6f service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.046361 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1398/2738] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:58.245286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.246891 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:58.324467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212c775-6175-41e2-b32f-c8d514637c71 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.329552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.330852 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.331328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.331450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] PATH_INFO: `/v3/projects/4c1c9729df694f46987e6dd30a09d5d2` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.332726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1c9729df694f46987e6dd30a09d5d2)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.335027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.386576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1c9729df694f46987e6dd30a09d5d2', 'request_id': 'req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:58.392757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1c9729df694f46987e6dd30a09d5d2', 'request_id': 'req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:58.407136 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1c9729df694f46987e6dd30a09d5d2', 'request_id': 'req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:58.422382 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] The token cache is being invalidate because project 4c1c9729df694f46987e6dd30a09d5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:46:58.422820 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:58.423659 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba17cb5e-0a1f-4151-a2aa-ba5d3f6a8ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:46:58.426284 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1341/2739] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:46:58 2022] DELETE /identity/v3/projects/4c1c9729df694f46987e6dd30a09d5d2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:58.432533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.433102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.433499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.514601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:58.515874 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:58.579286 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1399/2740] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:58.587916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.588282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.588646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.610105 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:58.611304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:58.619957 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1342/2741] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:58.624582 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.631126 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:58.635957 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7648c5-4270-4117-9d8e-a7e6f71b984b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.641185 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.641809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.642085 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.642353 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.643048 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:46:58.643348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.645452 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.661811 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1400/2742] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:46:58.663414 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.674418 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf148b3-4a67-4670-91f5-41fe952d8b42 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.680513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.681288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.681629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.681966 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.682969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.684958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.718143 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1343/2743] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:58 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:58.722791 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.735466 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b171c2-1774-4e18-a1b0-bcde9ee4cc9e admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.798051 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.798732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.799014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.799307 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.800028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.802029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.826952 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1401/2744] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:58 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:58.831227 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.842389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c9372-c6be-4084-b33b-1667715a4207 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.848012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.848666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.848956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.849441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.850162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.851806 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.863936 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1344/2745] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:58.868279 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.879446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e97c7-2c3e-422e-9fdc-02b7d5e186a0 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.884586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304837f2-053f-46c9-a414-acc249652804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.885217 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304837f2-053f-46c9-a414-acc249652804 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.885497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304837f2-053f-46c9-a414-acc249652804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.885771 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304837f2-053f-46c9-a414-acc249652804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.886469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304837f2-053f-46c9-a414-acc249652804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.888264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304837f2-053f-46c9-a414-acc249652804 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.901049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1402/2746] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:58.905579 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:58.912236 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:58.916771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639221ed-18bd-4cb9-8f58-99e5d4cbe340 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:58.922404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:58.923476 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.924129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.924410 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] PATH_INFO: `/v3/projects/a06d55f89ded452bac2f50cf5597616a/users/1081914058594cc7aa7b7f4782b976c0/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:58.942869 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06d55f89ded452bac2f50cf5597616a, user_id=1081914058594cc7aa7b7f4782b976c0, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:58.956189 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:58.983320 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1345/2747] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:58 2022] PUT /identity/v3/projects/a06d55f89ded452bac2f50cf5597616a/users/1081914058594cc7aa7b7f4782b976c0/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:46:58.994107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:58.994550 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:58.994935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.034601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] MFA Rules not processed for user `1081914058594cc7aa7b7f4782b976c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:59.036082 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:59.108613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1403/2748] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:59.114076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.196473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f0ee-0aae-49b4-b787-283ca28b5de0 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.202220 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.203804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.204092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.204634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.205851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.211109 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.257027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1346/2749] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:59.261104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.272406 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64c5b3-007b-4c7a-a987-1e5654568814 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.360778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.364397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.364703 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.364981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.365810 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.367918 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.377256 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:59.406283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1404/2750] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:59 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:59.407383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.419851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99535098-c678-41f0-b4ef-d41811134821 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.424636 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.425352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.425631 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.425902 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.426629 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.429110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.448154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1347/2751] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:59.452228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.469992 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5d162-9dd7-493c-9554-a06d3d0d5b99 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.475123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5a865-8b4b-4734-a8c6-f7eca1956b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.475792 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5a865-8b4b-4734-a8c6-f7eca1956b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.476067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5a865-8b4b-4734-a8c6-f7eca1956b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.476339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5a865-8b4b-4734-a8c6-f7eca1956b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.477055 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5a865-8b4b-4734-a8c6-f7eca1956b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.478672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5a865-8b4b-4734-a8c6-f7eca1956b12 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.492881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1405/2752] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:46:59.497275 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.508804 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bb340-9b93-4c62-88cf-561bc987aa59 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.513846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.514643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.514922 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.515315 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] PATH_INFO: `/v3/projects/10d55a765d3146c28ce0225a25b62d5a/users/e3a18615a57d4120afc35ef7272d4199/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.535028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d55a765d3146c28ce0225a25b62d5a, user_id=e3a18615a57d4120afc35ef7272d4199, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.537727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.552102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1348/2753] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:59 2022] PUT /identity/v3/projects/10d55a765d3146c28ce0225a25b62d5a/users/e3a18615a57d4120afc35ef7272d4199/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:46:59.564304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.564574 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.564844 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.595586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] MFA Rules not processed for user `e3a18615a57d4120afc35ef7272d4199`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:59.596963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:59.669708 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1406/2754] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:46:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:59.674489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.741828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c106545-ced8-4d02-969f-103bc576f3ed admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.746684 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.747308 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.747582 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.747893 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.748622 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.750475 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.781856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1349/2755] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:59.786365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.797407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ba571-f526-4220-a0f9-71d90530a8e8 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.865092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.865738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.866016 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.866284 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.866964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.868982 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.899363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1407/2756] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:46:59 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:46:59.904088 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.915511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b497f4-d473-4086-b7dd-1e6d7293eb34 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.920478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.921121 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.921422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.921689 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.922396 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.924066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.936122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1350/2757] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:46:59.940714 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:59.952095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83717ad-b09b-4f67-bb45-84c587cdbb89 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:59.958132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:59.958829 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:59.959108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:59.959713 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] PATH_INFO: `/v3/projects/948d5b55ff074357a7b1c6645c7345e7/users/524a372de484419aba28048f9d0169ec/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:59.978073 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948d5b55ff074357a7b1c6645c7345e7, user_id=524a372de484419aba28048f9d0169ec, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:59.982028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:59.997329 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1408/2758] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:46:59 2022] PUT /identity/v3/projects/948d5b55ff074357a7b1c6645c7345e7/users/524a372de484419aba28048f9d0169ec/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:00.001548 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:00.008989 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:00.015910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74539017-6723-4c68-a802-76ca034cf96e admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:00.087291 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d38880-15a5-488f-8794-1eb29ad5943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:00.087984 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d38880-15a5-488f-8794-1eb29ad5943c admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.088259 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d38880-15a5-488f-8794-1eb29ad5943c admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.088521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d38880-15a5-488f-8794-1eb29ad5943c admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.090791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d38880-15a5-488f-8794-1eb29ad5943c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:00.092540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d38880-15a5-488f-8794-1eb29ad5943c admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:00.103955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1351/2759] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:00.108625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:00.121129 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38734656-6a14-46af-8288-52ec867b5947 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:00.126334 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:00.127032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.127310 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.127580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] PATH_INFO: `/v3/projects/948d5b55ff074357a7b1c6645c7345e7/users/524a372de484419aba28048f9d0169ec/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.130340 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] RBAC: Authorizing `identity:create_grant(project_id=948d5b55ff074357a7b1c6645c7345e7, user_id=524a372de484419aba28048f9d0169ec, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:00.133368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:00.146864 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1409/2760] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:47:00 2022] PUT /identity/v3/projects/948d5b55ff074357a7b1c6645c7345e7/users/524a372de484419aba28048f9d0169ec/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:00.157663 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa097666-2608-4b52-8411-9efafc76251e None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.157956 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa097666-2608-4b52-8411-9efafc76251e None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.158232 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa097666-2608-4b52-8411-9efafc76251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.199509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-aa097666-2608-4b52-8411-9efafc76251e None None] MFA Rules not processed for user `524a372de484419aba28048f9d0169ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:00.201263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa097666-2608-4b52-8411-9efafc76251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:00.283647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1352/2761] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:00.296605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:00.298717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:00.323672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:00.367153 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02fcb-cff3-4e61-88a7-0904e2b1440d admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:00.371613 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1405e-da44-4849-b88a-038025475e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:00.372412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1405e-da44-4849-b88a-038025475e58 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.372932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1405e-da44-4849-b88a-038025475e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.373304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1405e-da44-4849-b88a-038025475e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.383941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1405e-da44-4849-b88a-038025475e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:00.385393 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1405e-da44-4849-b88a-038025475e58 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:00.396890 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1410/2762] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:00.561314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa097666-2608-4b52-8411-9efafc76251e None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:00.565369 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-aa097666-2608-4b52-8411-9efafc76251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:00.655927 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa097666-2608-4b52-8411-9efafc76251e None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:00.665039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:00.665039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.665039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.665039 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] PATH_INFO: `/v3/users/52ad0bd481a846f6b5c807fd69904201` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.666500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ad0bd481a846f6b5c807fd69904201)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:00.668635 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:00.710204 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:00.733432 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ad0bd481a846f6b5c807fd69904201'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:00.734229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ad0bd481a846f6b5c807fd69904201'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:00.749060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ad0bd481a846f6b5c807fd69904201'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:00.759054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ad0bd481a846f6b5c807fd69904201'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:00.759837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ad0bd481a846f6b5c807fd69904201'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:00.784017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1353/2763] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:47:00 2022] DELETE /identity/v3/users/52ad0bd481a846f6b5c807fd69904201 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:00.789851 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.790487 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.790832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.819392 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:00.821586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:00.923686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1411/2764] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:00.934420 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:00.936049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.014165 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d629b-bb67-462a-ae2a-0728f97060d3 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.019267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.020261 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.020632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.020910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.022405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.024031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.032856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.034586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.045399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1354/2765] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:01.101343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b61dbc-e890-43c8-8dfa-1f5a6cac7938 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.105656 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.106302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.106580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.106899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.109357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.194604 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.196648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df39f2-546a-4e67-be9e-7ab46d08f271 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.207388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1412/2766] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:47:01.391848 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.394042 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.410557 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0188e-f0fa-4b0d-8df9-6779b1ab404e service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.416197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.417081 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.417351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.417621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] PATH_INFO: `/v3/projects/7599805e90374ddbbd17bf9899bdc8b2` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.419126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7599805e90374ddbbd17bf9899bdc8b2)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.421129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.479053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7599805e90374ddbbd17bf9899bdc8b2', 'request_id': 'req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:01.484951 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7599805e90374ddbbd17bf9899bdc8b2', 'request_id': 'req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:01.504551 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7599805e90374ddbbd17bf9899bdc8b2', 'request_id': 'req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:01.518201 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] The token cache is being invalidate because project 7599805e90374ddbbd17bf9899bdc8b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:47:01.518634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:01.520133 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-12ddd2a5-e794-44cf-82ed-6d6c07d79bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:01.522072 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1355/2767] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:47:01 2022] DELETE /identity/v3/projects/7599805e90374ddbbd17bf9899bdc8b2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:01.529094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.529378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.529660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.553209 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:01.554499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.580269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=100 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:01.636261 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1413/2768] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:01.644916 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.645217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.645473 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.666389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:01.667656 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.676678 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1356/2769] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:47:01.680842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.691004 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc8bff-83e5-4704-8ea7-a236b6aac18a None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.695718 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.696372 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.696644 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.696909 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.697629 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:47:01.697927 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.699532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.712175 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1414/2770] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:47:01.716818 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.726856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce363a1-2781-438a-a2a1-166536dc8d30 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.732644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.733412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.733687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.733967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.734634 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.736589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.766824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1357/2771] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:47:01 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:47:01.771490 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.783367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cff4b-bbeb-4684-896c-1fcfa1fa6a36 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.843190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.843901 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.844108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.844377 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.845095 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.847374 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.870341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1415/2772] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:47:01 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:47:01.875028 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.880801 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:01.886898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51025ae1-2384-40bc-9ae5-90c68f7386d1 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.893199 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.895269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.895269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.895269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.898281 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.898385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.913602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1358/2773] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:01.915020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.926319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b1ca1-d4dc-4de6-87e5-b8f6ae3a91e8 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.932985 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5503c11-d69f-4a58-b6e5-5ba8d924dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.933925 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503c11-d69f-4a58-b6e5-5ba8d924dab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.934470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503c11-d69f-4a58-b6e5-5ba8d924dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.934753 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503c11-d69f-4a58-b6e5-5ba8d924dab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.935462 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5503c11-d69f-4a58-b6e5-5ba8d924dab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.937881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5503c11-d69f-4a58-b6e5-5ba8d924dab9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.950945 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1416/2774] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:01.955562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:01.968918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848e3e8-85c6-48fe-af71-6ef0a2eba182 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:01.975296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:01.976013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:01.976288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:01.976555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] PATH_INFO: `/v3/projects/06b543bebc784ab89f7ac38d220da11a/users/6785d7f003f540f782c70e4af1a4600e/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:01.995658 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b543bebc784ab89f7ac38d220da11a, user_id=6785d7f003f540f782c70e4af1a4600e, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.998431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:02.014732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1359/2775] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:47:01 2022] PUT /identity/v3/projects/06b543bebc784ab89f7ac38d220da11a/users/6785d7f003f540f782c70e4af1a4600e/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:02.027728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:02.028038 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:02.028344 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:02.062190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] MFA Rules not processed for user `6785d7f003f540f782c70e4af1a4600e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:02.063455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:02.139311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1417/2776] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:02.151257 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:02.153510 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:02.222491 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e547-0481-4dc5-ac3c-6e7b6e757481 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:02.226361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:02.227005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:02.227287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:02.227589 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:02.238412 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:02.239941 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:02.249361 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1360/2777] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:04.100385 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:04.101757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:04.131869 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:04.207419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df511d-df23-4a18-bfd1-a6392195c167 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:04.213396 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:04.214032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:04.214308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:04.214575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:04.216434 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:04.229845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:04.231455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:04.242469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1418/2778] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:07.522561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:07.532595 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992b761-5301-49ca-ba13-b68ebc7a96cf service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:07.538079 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:07.538745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:07.539237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:07.539569 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:07.541945 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:07.619624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:07.621152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c817d4-bff5-439d-931f-576061a93300 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:07.632321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1361/2779] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:13.013192 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:13.014896 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:13.097880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efdb9e-c5a3-4612-9957-017b163ad6d8 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:13.103566 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:13.104262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:13.104534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:13.104783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] PATH_INFO: `/v3/users/1081914058594cc7aa7b7f4782b976c0` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:13.106099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1081914058594cc7aa7b7f4782b976c0)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:13.108341 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:13.160748 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1081914058594cc7aa7b7f4782b976c0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.161402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1081914058594cc7aa7b7f4782b976c0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.170878 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1081914058594cc7aa7b7f4782b976c0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.185084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1081914058594cc7aa7b7f4782b976c0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.192449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1081914058594cc7aa7b7f4782b976c0'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.199068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1419/2780] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:47:13 2022] DELETE /identity/v3/users/1081914058594cc7aa7b7f4782b976c0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:13.203851 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:13.204384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:13.204564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:13.227522 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:13.228788 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:13.291122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1362/2781] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:13.301299 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:13.302769 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:13.330345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:13.386378 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac66dc-e050-4b2d-9565-59ffe941a7a6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:13.390015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:13.390633 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:13.390905 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:13.391169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:13.401542 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:13.402975 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:13.414008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1420/2782] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:13.728824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:13.730426 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:13.744383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90bf09-6514-46be-b21d-2aa65b161b03 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:13.749827 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:13.750511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:13.750786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:13.751056 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] PATH_INFO: `/v3/projects/a06d55f89ded452bac2f50cf5597616a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:13.752383 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06d55f89ded452bac2f50cf5597616a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:13.755129 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:13.758241 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:13.809258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06d55f89ded452bac2f50cf5597616a', 'request_id': 'req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.815511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06d55f89ded452bac2f50cf5597616a', 'request_id': 'req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.824521 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06d55f89ded452bac2f50cf5597616a', 'request_id': 'req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.838151 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] The token cache is being invalidate because project a06d55f89ded452bac2f50cf5597616a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:47:13.838439 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.839107 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.840727 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1363/2783] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:47:13 2022] DELETE /identity/v3/projects/a06d55f89ded452bac2f50cf5597616a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:13.845020 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:13.855537 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee787b4-664b-4266-b93d-3b5698dd68ab service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:13.919671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:13.920258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:13.920535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:13.920808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] PATH_INFO: `/v3/users/e3a18615a57d4120afc35ef7272d4199` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:13.922033 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a18615a57d4120afc35ef7272d4199)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:13.924009 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:13.976841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a18615a57d4120afc35ef7272d4199'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.977534 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a18615a57d4120afc35ef7272d4199'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.985668 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a18615a57d4120afc35ef7272d4199'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:13.998238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a18615a57d4120afc35ef7272d4199'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.006143 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a18615a57d4120afc35ef7272d4199'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.012407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1421/2784] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:47:13 2022] DELETE /identity/v3/users/e3a18615a57d4120afc35ef7272d4199 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:14.294974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:14.296501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:14.313032 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729b8a-ead0-4933-a72d-0f38c5c3e1ea admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:14.379118 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:14.379821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:14.380110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:14.380390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] PATH_INFO: `/v3/projects/10d55a765d3146c28ce0225a25b62d5a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:14.381632 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d55a765d3146c28ce0225a25b62d5a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:14.383602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:14.435187 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d55a765d3146c28ce0225a25b62d5a', 'request_id': 'req-2069af33-87d8-4f39-bc40-c930aa86dfbf'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.441670 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d55a765d3146c28ce0225a25b62d5a', 'request_id': 'req-2069af33-87d8-4f39-bc40-c930aa86dfbf'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.450780 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d55a765d3146c28ce0225a25b62d5a', 'request_id': 'req-2069af33-87d8-4f39-bc40-c930aa86dfbf'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.463687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] The token cache is being invalidate because project 10d55a765d3146c28ce0225a25b62d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:47:14.464096 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.464861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.466497 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1364/2785] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:47:14 2022] DELETE /identity/v3/projects/10d55a765d3146c28ce0225a25b62d5a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:14.470937 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:14.473157 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:14.487270 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d39a-bfe3-4e8d-bdd8-ddcf34da2c2c admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:14.510136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:14.551208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:14.551900 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:14.552180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:14.552449 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] PATH_INFO: `/v3/users/524a372de484419aba28048f9d0169ec` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:14.553698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] RBAC: Authorizing `identity:delete_user(user_id=524a372de484419aba28048f9d0169ec)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:14.555659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:14.612018 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524a372de484419aba28048f9d0169ec'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.612916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524a372de484419aba28048f9d0169ec'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.623847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524a372de484419aba28048f9d0169ec'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.638363 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524a372de484419aba28048f9d0169ec'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.645847 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-0b134aac-5601-4c8f-80d2-8aa30973053d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524a372de484419aba28048f9d0169ec'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:14.653169 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1422/2786] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:47:14 2022] DELETE /identity/v3/users/524a372de484419aba28048f9d0169ec => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:14.965053 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:14.966578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:14.984400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069af33-87d8-4f39-bc40-c930aa86dfbf admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.007466 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:15.051699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.052509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.052680 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.052924 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] PATH_INFO: `/v3/projects/948d5b55ff074357a7b1c6645c7345e7` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.054185 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=948d5b55ff074357a7b1c6645c7345e7)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.056124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.107701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948d5b55ff074357a7b1c6645c7345e7', 'request_id': 'req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:15.113692 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948d5b55ff074357a7b1c6645c7345e7', 'request_id': 'req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:15.122917 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948d5b55ff074357a7b1c6645c7345e7', 'request_id': 'req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:15.136126 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] The token cache is being invalidate because project 948d5b55ff074357a7b1c6645c7345e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:47:15.136393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:15.137045 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-b722f2b2-8a1a-47e9-bdfe-276a80c05eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:15.138593 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1365/2787] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:47:14 2022] DELETE /identity/v3/projects/948d5b55ff074357a7b1c6645c7345e7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:15.145115 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.145426 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.145705 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.166304 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:15.167520 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:15.236328 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1423/2788] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:15.245954 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183746-e62b-489c-972a-302a745f3306 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.246206 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183746-e62b-489c-972a-302a745f3306 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.246482 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183746-e62b-489c-972a-302a745f3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.268878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-d6183746-e62b-489c-972a-302a745f3306 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:15.270100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-d6183746-e62b-489c-972a-302a745f3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:15.279286 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1366/2789] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:15.285557 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.296944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd54e5-aeb3-4d69-93ae-a1f7ce718361 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.303066 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.303568 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.303907 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.304294 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.305200 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:47:15.305491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.307554 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.323828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1424/2790] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:47:15.328449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183746-e62b-489c-972a-302a745f3306 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.339394 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183746-e62b-489c-972a-302a745f3306 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.345907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.346602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.347066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.347066 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.348051 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.350017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.385085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1367/2791] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:47:15 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:47:15.391942 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.405364 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f9e0-1277-4c7c-a51f-2613624f39c6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.439133 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:15.470415 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.471050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.471729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.471729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.472704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.474820 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.511475 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1425/2792] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:47:15 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:47:15.515946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.526946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476128e8-91ed-4131-8e20-1846939e0740 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.532312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.532977 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.534171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.534171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.534552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.536282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.547197 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1368/2793] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:15.551998 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.563352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e76723-fcce-460a-b01d-1a92358f644f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.568964 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b602-c071-4a6d-a201-676764b80414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.569660 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b602-c071-4a6d-a201-676764b80414 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.569968 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b602-c071-4a6d-a201-676764b80414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.570285 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b602-c071-4a6d-a201-676764b80414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.571043 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b602-c071-4a6d-a201-676764b80414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.572788 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b602-c071-4a6d-a201-676764b80414 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.584417 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1426/2794] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:15.589726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.601558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed76ad0-04eb-41ee-b8da-022103386eb6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.608517 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.609208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.609485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.609747 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] PATH_INFO: `/v3/projects/f7d6165b87444968b256ca7d7feed3b9/users/952b76e436134aa49f8960ff53170422/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.627654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d6165b87444968b256ca7d7feed3b9, user_id=952b76e436134aa49f8960ff53170422, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.630441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.647464 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1369/2795] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:47:15 2022] PUT /identity/v3/projects/f7d6165b87444968b256ca7d7feed3b9/users/952b76e436134aa49f8960ff53170422/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:15.660903 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.661078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.661325 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.702533 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] MFA Rules not processed for user `952b76e436134aa49f8960ff53170422`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:15.703733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:15.774984 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1427/2796] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:15.786910 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.788839 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:15.848506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:15.852759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a234b5-60a1-4361-9bb0-6cf105399654 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.857399 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2f993-bfe2-4165-9c9f-27e74d7fb83f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.858037 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2f993-bfe2-4165-9c9f-27e74d7fb83f service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.858660 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2f993-bfe2-4165-9c9f-27e74d7fb83f service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.858964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2f993-bfe2-4165-9c9f-27e74d7fb83f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.869108 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2f993-bfe2-4165-9c9f-27e74d7fb83f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.870586 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2f993-bfe2-4165-9c9f-27e74d7fb83f service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:15.880362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1370/2797] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:15.906408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:15.969783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4edcfd-24d8-4b5e-a305-9d7534f0022f None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:15.975215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:15.975916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:15.976164 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:15.976457 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] PATH_INFO: `/v3/users/952b76e436134aa49f8960ff53170422` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:15.977649 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952b76e436134aa49f8960ff53170422)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:15.979673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:16.030485 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:16.036390 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952b76e436134aa49f8960ff53170422'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.037096 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952b76e436134aa49f8960ff53170422'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.045795 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952b76e436134aa49f8960ff53170422'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.058631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952b76e436134aa49f8960ff53170422'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.066513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952b76e436134aa49f8960ff53170422'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.072297 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1428/2798] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:47:15 2022] DELETE /identity/v3/users/952b76e436134aa49f8960ff53170422 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:16.077013 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.078757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.078757 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.102921 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:16.104067 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:16.169200 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1371/2799] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:47:16.179624 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:16.181194 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:16.243102 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61979e-adbd-4418-b983-79da39094901 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:16.247104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788b75f-5700-432a-816a-1d412e5a7cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:16.247728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b75f-5700-432a-816a-1d412e5a7cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.248037 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b75f-5700-432a-816a-1d412e5a7cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.248305 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788b75f-5700-432a-816a-1d412e5a7cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.258659 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788b75f-5700-432a-816a-1d412e5a7cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:16.260146 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788b75f-5700-432a-816a-1d412e5a7cbe service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:16.270235 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1429/2800] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:16.591578 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:16.593120 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:16.606139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3af3f-6a55-4116-b2eb-ae3eab58cf9f None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:16.613347 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:16.614398 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.614814 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.614964 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] PATH_INFO: `/v3/projects/f7d6165b87444968b256ca7d7feed3b9` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.616422 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d6165b87444968b256ca7d7feed3b9)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:16.618373 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:16.684701 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d6165b87444968b256ca7d7feed3b9', 'request_id': 'req-ba88f285-e693-4656-8e92-7132f7b5bf63'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.691784 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d6165b87444968b256ca7d7feed3b9', 'request_id': 'req-ba88f285-e693-4656-8e92-7132f7b5bf63'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.701148 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d6165b87444968b256ca7d7feed3b9', 'request_id': 'req-ba88f285-e693-4656-8e92-7132f7b5bf63'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.716324 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] The token cache is being invalidate because project f7d6165b87444968b256ca7d7feed3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:47:16.716708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.717278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ba88f285-e693-4656-8e92-7132f7b5bf63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:16.719238 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1372/2801] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:47:16 2022] DELETE /identity/v3/projects/f7d6165b87444968b256ca7d7feed3b9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:16.727212 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.727687 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.728215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.751013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:16.752527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:16.824101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1430/2802] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:16.832258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.832742 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.833078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.860639 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:16.861991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:16.871378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1373/2803] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:16.875966 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:16.887288 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d16f4c-e16c-402f-957e-37b302b90a2e None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:16.892269 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:16.892913 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.893190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.893778 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.894701 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80349) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:47:16.895078 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:16.897077 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:16.910584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1431/2804] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:47:16.915288 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:16.925027 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3edd1-9cb5-41cd-a66f-acd5a8895c1d None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:16.930245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:16.930863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:16.931123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:16.931400 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:16.932111 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:16.934044 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:16.965894 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1374/2805] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:47:16 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:47:16.970093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:16.980601 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a7ee7-3ca1-4039-8d75-1c16fcc0aa32 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:17.034459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:17.051104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:17.051652 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:17.051963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:17.052229 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:17.053068 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:17.055312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:17.083249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1432/2806] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:47:16 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:47:17.084918 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:17.098506 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce34b0-f134-4e7e-aa48-96a0afabba7d admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:17.103293 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:17.103974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:17.104256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:17.104526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:17.105233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:17.106863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:17.119459 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1375/2807] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:17.123672 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:17.136387 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c75a0-407d-4175-8f79-8a9cf041a3eb admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:17.141704 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012427a-0e81-4453-a0cf-4607e4f5f15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:17.142230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012427a-0e81-4453-a0cf-4607e4f5f15c admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:17.142499 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012427a-0e81-4453-a0cf-4607e4f5f15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:17.142783 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012427a-0e81-4453-a0cf-4607e4f5f15c admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:17.143549 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012427a-0e81-4453-a0cf-4607e4f5f15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:17.145318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012427a-0e81-4453-a0cf-4607e4f5f15c admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:17.157006 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1433/2808] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:47:17.161263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:17.172601 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3643cb-4b87-4938-a60f-7281ec1c4fff admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:17.178033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:17.178812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:17.179092 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:17.179359 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] PATH_INFO: `/v3/projects/5ac43ede7dc24773ba18e325f9f1f631/users/62cb33751233433a8670cd52f9583bed/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:17.196450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac43ede7dc24773ba18e325f9f1f631, user_id=62cb33751233433a8670cd52f9583bed, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:17.200405 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:17.217203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1376/2809] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:47:17 2022] PUT /identity/v3/projects/5ac43ede7dc24773ba18e325f9f1f631/users/62cb33751233433a8670cd52f9583bed/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:17.228757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:17.229172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:17.229468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:17.262202 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] MFA Rules not processed for user `62cb33751233433a8670cd52f9583bed`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:17.263455 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:17.333876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1434/2810] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:17.345564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:17.347574 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:17.362362 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:17.441393 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937d1c-81e2-4f88-9ab1-39bdc59fe134 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:17.445411 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba186cf-4c94-40fb-bd02-5cae7cec07fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:17.446048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba186cf-4c94-40fb-bd02-5cae7cec07fb service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:17.446332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba186cf-4c94-40fb-bd02-5cae7cec07fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:17.446592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba186cf-4c94-40fb-bd02-5cae7cec07fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:17.455602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba186cf-4c94-40fb-bd02-5cae7cec07fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:17.457360 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba186cf-4c94-40fb-bd02-5cae7cec07fb service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:17.467994 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1377/2811] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:21.872600 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:21.874322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:21.947223 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d3faf-9129-48cc-9a54-73bb93d9255b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:21.952764 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:21.953438 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:21.953762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:21.954028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] PATH_INFO: `/v3/users/6785d7f003f540f782c70e4af1a4600e` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:21.955445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6785d7f003f540f782c70e4af1a4600e)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:21.957648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:22.011785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6785d7f003f540f782c70e4af1a4600e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.012465 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6785d7f003f540f782c70e4af1a4600e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.020318 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6785d7f003f540f782c70e4af1a4600e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.026768 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:22.035362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6785d7f003f540f782c70e4af1a4600e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.043190 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6785d7f003f540f782c70e4af1a4600e'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.049648 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1435/2812] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:47:21 2022] DELETE /identity/v3/users/6785d7f003f540f782c70e4af1a4600e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:47:22.054345 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:22.055055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:22.055055 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:22.075654 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:22.076940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:22.137356 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1378/2813] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:22.147631 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:22.149348 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:22.211098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68c72-7d25-4bdc-b5fa-e12becbe2eb7 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:22.214845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573e844-59f7-4db6-a50f-aa2be6563257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:22.215464 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e844-59f7-4db6-a50f-aa2be6563257 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:22.215738 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e844-59f7-4db6-a50f-aa2be6563257 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:22.216032 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e844-59f7-4db6-a50f-aa2be6563257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:22.225756 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573e844-59f7-4db6-a50f-aa2be6563257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:22.227204 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573e844-59f7-4db6-a50f-aa2be6563257 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:22.236793 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1436/2814] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:22.531697 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:22.533321 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:22.546433 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3328b2-c7aa-4c88-b685-bb062d9d60af None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:22.552474 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:22.553145 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:22.553421 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:22.553681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] PATH_INFO: `/v3/projects/06b543bebc784ab89f7ac38d220da11a` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:22.554981 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b543bebc784ab89f7ac38d220da11a)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:22.557229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:22.609078 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b543bebc784ab89f7ac38d220da11a', 'request_id': 'req-768e5289-ff4b-4317-b6d5-f0f297477507'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.614786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b543bebc784ab89f7ac38d220da11a', 'request_id': 'req-768e5289-ff4b-4317-b6d5-f0f297477507'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.624390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b543bebc784ab89f7ac38d220da11a', 'request_id': 'req-768e5289-ff4b-4317-b6d5-f0f297477507'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.636881 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] The token cache is being invalidate because project 06b543bebc784ab89f7ac38d220da11a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:47:22.637149 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.637874 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:47:22.639435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1379/2815] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:47:22 2022] DELETE /identity/v3/projects/06b543bebc784ab89f7ac38d220da11a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:47:32.232597 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=2,keystone:SELECT=35 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:32.633768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:56.323789 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a042b44-f7db-4513-9714-51c24415dba5 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:56.324425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a042b44-f7db-4513-9714-51c24415dba5 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:56.324425 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a042b44-f7db-4513-9714-51c24415dba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:56.345418 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-6a042b44-f7db-4513-9714-51c24415dba5 None None] MFA Rules not processed for user `39593dbdff134fb1af13b341d6c01e37`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:56.346655 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-6a042b44-f7db-4513-9714-51c24415dba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:56.408800 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1437/2816] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:01.645467 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:01.647558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:01.719726 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e5289-ff4b-4317-b6d5-f0f297477507 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:01.724895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:01.725248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:01.725520 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:01.725779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] PATH_INFO: `/v3/users/62cb33751233433a8670cd52f9583bed` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:01.727104 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=62cb33751233433a8670cd52f9583bed)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:01.729076 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:01.784100 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62cb33751233433a8670cd52f9583bed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:01.785060 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62cb33751233433a8670cd52f9583bed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:01.799484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62cb33751233433a8670cd52f9583bed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:01.808602 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62cb33751233433a8670cd52f9583bed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:01.809378 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62cb33751233433a8670cd52f9583bed'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:01.826809 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1380/2817] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:48:01 2022] DELETE /identity/v3/users/62cb33751233433a8670cd52f9583bed => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:01.832195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:01.832751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:01.832751 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:01.854643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:01.855932 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:01.918319 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1438/2818] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:01.929161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:01.930715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:01.992716 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076c2f-98f0-4ae2-99b2-cd7045cb83fa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:01.998896 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70295708-69c7-41db-a9f7-b731e87f5eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:01.999650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295708-69c7-41db-a9f7-b731e87f5eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:01.999953 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295708-69c7-41db-a9f7-b731e87f5eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.000222 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295708-69c7-41db-a9f7-b731e87f5eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.012486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70295708-69c7-41db-a9f7-b731e87f5eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.014019 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70295708-69c7-41db-a9f7-b731e87f5eb1 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.025106 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1381/2819] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:02.325407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.327405 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:02.343497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2102e-380a-4a49-b454-0f8e6d1614e1 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.348238 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.348944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.349215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.349468 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] PATH_INFO: `/v3/projects/5ac43ede7dc24773ba18e325f9f1f631` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.350698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac43ede7dc24773ba18e325f9f1f631)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.352828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.416193 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac43ede7dc24773ba18e325f9f1f631', 'request_id': 'req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:02.426248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac43ede7dc24773ba18e325f9f1f631', 'request_id': 'req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:02.435944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac43ede7dc24773ba18e325f9f1f631', 'request_id': 'req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:02.451524 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] The token cache is being invalidate because project 5ac43ede7dc24773ba18e325f9f1f631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:48:02.451947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:02.452760 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-a607185d-b94f-4ffd-ae9d-2f68d9291d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:02.454515 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1439/2820] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:48:02 2022] DELETE /identity/v3/projects/5ac43ede7dc24773ba18e325f9f1f631 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:02.460509 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.460911 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.461243 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.484256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:02.485591 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:02.504750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:02.550094 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1382/2821] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:02.558467 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.558775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.559025 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.580608 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:02.581825 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:02.591794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1440/2822] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:02.596992 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.607214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451786aa-8333-4d07-97f0-314ecdb6d15a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.612508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.613158 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.613431 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.613811 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.614564 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:48:02.614880 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.616611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.630349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1383/2823] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:48:02.634974 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.645257 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baeac8-af2d-476b-a0ac-d3a7e57c5605 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.650532 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.651172 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.651447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.651706 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.652419 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.654358 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.684650 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:02.692832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1441/2824] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:48:02 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:48:02.697681 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.708312 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedf77b-3b13-42e6-87c4-5b96efbe932f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.773898 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.774540 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.774819 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.775091 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.775815 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.777847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.801265 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1384/2825] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:48:02 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:48:02.806460 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.820201 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bdef0-ae0f-409b-80f0-ddc8070e92c5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.825498 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.826132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.826413 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.826683 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.827408 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.829084 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.840855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1442/2826] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:48:02.845082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.856978 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18aa48-2677-42b0-9d50-42f35d7ce910 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.863021 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7029df-eb59-477c-aeff-b6b62d321cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.864009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7029df-eb59-477c-aeff-b6b62d321cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.864207 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7029df-eb59-477c-aeff-b6b62d321cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.864672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7029df-eb59-477c-aeff-b6b62d321cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.865657 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7029df-eb59-477c-aeff-b6b62d321cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.867340 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7029df-eb59-477c-aeff-b6b62d321cfc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.889449 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1385/2827] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:48:02.894007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:02.905407 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12655c56-7bec-486c-b1ce-450a12d769ca admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:02.910841 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:02.911580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.912127 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.912397 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] PATH_INFO: `/v3/projects/9705a62930e2454d936da56ce1192613/users/398b30cc849449f7808fecbc43bc164c/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.930921 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9705a62930e2454d936da56ce1192613, user_id=398b30cc849449f7808fecbc43bc164c, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:02.933830 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:02.950876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1443/2828] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:48:02 2022] PUT /identity/v3/projects/9705a62930e2454d936da56ce1192613/users/398b30cc849449f7808fecbc43bc164c/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:02.963391 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:02.963728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:02.964017 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:02.996728 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] MFA Rules not processed for user `398b30cc849449f7808fecbc43bc164c`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:02.998267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:03.077789 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1386/2829] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:03.089691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:03.091430 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:03.150761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641cd72-e191-4fe2-9560-a388b1131118 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:03.154614 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ee2b-33b2-402f-85a7-02bd5d905041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:03.155271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ee2b-33b2-402f-85a7-02bd5d905041 service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:03.155547 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ee2b-33b2-402f-85a7-02bd5d905041 service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:03.155852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ee2b-33b2-402f-85a7-02bd5d905041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:03.164671 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1ee2b-33b2-402f-85a7-02bd5d905041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:03.166275 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1ee2b-33b2-402f-85a7-02bd5d905041 service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:03.176060 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1444/2830] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:03.200745 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:03.222141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:03.269450 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34bd0c-f327-43d4-8eb3-0036c2e3bad5 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:03.274912 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:03.275571 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:03.275863 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:03.276171 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] PATH_INFO: `/v3/users/398b30cc849449f7808fecbc43bc164c` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:03.277379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=398b30cc849449f7808fecbc43bc164c)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:03.279374 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:03.332513 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398b30cc849449f7808fecbc43bc164c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.333208 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398b30cc849449f7808fecbc43bc164c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.346162 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398b30cc849449f7808fecbc43bc164c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.353346 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398b30cc849449f7808fecbc43bc164c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.354050 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398b30cc849449f7808fecbc43bc164c'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.370237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1387/2831] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:48:03 2022] DELETE /identity/v3/users/398b30cc849449f7808fecbc43bc164c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:03.374893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:03.375173 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:03.375444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:03.395924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:03.397335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:03.464987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1445/2832] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:03.475046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:03.476647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:03.536098 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aeb96-909b-4049-bb9e-6d6d608d29bc admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:03.541061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076427d-2d6b-4041-9901-e725aa3aad5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:03.541666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076427d-2d6b-4041-9901-e725aa3aad5e service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:03.541946 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076427d-2d6b-4041-9901-e725aa3aad5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:03.542214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076427d-2d6b-4041-9901-e725aa3aad5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:03.552823 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076427d-2d6b-4041-9901-e725aa3aad5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:03.554390 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076427d-2d6b-4041-9901-e725aa3aad5e service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:03.565112 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1388/2833] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:03.870459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:03.872107 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:03.887029 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc08d0-40fb-4893-a943-255d7b0d72a3 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:03.892745 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:03.893950 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:03.894411 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:03.894691 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] PATH_INFO: `/v3/projects/9705a62930e2454d936da56ce1192613` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:03.896195 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9705a62930e2454d936da56ce1192613)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:03.898491 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:03.954081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9705a62930e2454d936da56ce1192613', 'request_id': 'req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.960973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9705a62930e2454d936da56ce1192613', 'request_id': 'req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.964785 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:03.970801 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9705a62930e2454d936da56ce1192613', 'request_id': 'req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.983749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] The token cache is being invalidate because project 9705a62930e2454d936da56ce1192613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:48:03.983981 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.984627 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-741f28f4-edcc-43cb-a2c7-d864ba2a3d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:03.986264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1446/2834] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:48:03 2022] DELETE /identity/v3/projects/9705a62930e2454d936da56ce1192613 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:03.991562 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:03.992122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:03.992122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.012998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:04.014229 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:04.055423 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:04.085650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1389/2835] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:48:04.093780 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.094081 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.094331 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.116827 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:04.118155 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:04.126580 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1447/2836] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:04.131301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.141489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852295-3218-430e-8b2b-1675c6829e3b None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.146768 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.147401 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.147696 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.147969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.148672 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:48:04.148973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.150581 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.163404 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1390/2837] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:48:04.168083 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.177346 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d20ef3-348c-44b6-b088-89e837ae082b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.182094 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.182860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.182991 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.183241 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.184046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.185947 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.216362 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1448/2838] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:48:04 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:48:04.230033 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.240798 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2e351-712c-4bb4-94c5-5ab8a8fe9deb admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.302314 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.303264 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.303552 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.303650 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.304333 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.306279 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.329987 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1391/2839] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:48:04 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:48:04.334625 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.346607 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795de302-9ed1-40b9-ac6e-e8f4c412d4fe admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.352416 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.353104 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.353530 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.353882 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.354920 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.356447 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.368290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1449/2840] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:48:04.372550 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.383267 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5202fd4-7144-48e9-9af0-d0d23c6ea243 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.388974 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfb0d-de06-48a3-96a9-972fd15ee6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.389771 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfb0d-de06-48a3-96a9-972fd15ee6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.390214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfb0d-de06-48a3-96a9-972fd15ee6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.390489 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfb0d-de06-48a3-96a9-972fd15ee6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.391687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebfb0d-de06-48a3-96a9-972fd15ee6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.393152 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebfb0d-de06-48a3-96a9-972fd15ee6f1 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.410010 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1392/2841] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:48:04.411832 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.421761 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779f6d2-7ea2-4e26-a01b-c06b411ae01f admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.429110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.430156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.430302 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.430609 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] PATH_INFO: `/v3/projects/b05f2e96198f4abf98e10e1740149240/users/f1daaa8d84b7464885cd824a231b851f/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.450535 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05f2e96198f4abf98e10e1740149240, user_id=f1daaa8d84b7464885cd824a231b851f, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.453543 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.471470 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1450/2842] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:48:04 2022] PUT /identity/v3/projects/b05f2e96198f4abf98e10e1740149240/users/f1daaa8d84b7464885cd824a231b851f/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:48:04.482687 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.482975 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.483248 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.516867 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] MFA Rules not processed for user `f1daaa8d84b7464885cd824a231b851f`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:04.518110 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:04.585468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1393/2843] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:04.597973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:04.600289 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:04.660808 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa53d6-189d-44d7-a7b5-76ec13621ad5 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:04.664944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af15ce0-7663-45a7-88aa-bf6ee44d827a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:04.665558 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15ce0-7663-45a7-88aa-bf6ee44d827a service cinder] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:04.665828 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15ce0-7663-45a7-88aa-bf6ee44d827a service cinder] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:04.666092 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15ce0-7663-45a7-88aa-bf6ee44d827a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:04.675420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af15ce0-7663-45a7-88aa-bf6ee44d827a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:04.676944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af15ce0-7663-45a7-88aa-bf6ee44d827a service cinder] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:04.687086 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1451/2844] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:11.825342 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:11.867020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:11.893328 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483a86a-1c5a-4760-878f-e9123a2fcf22 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:11.898296 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:11.898958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:11.899263 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:11.899526 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] PATH_INFO: `/v3/users/f1daaa8d84b7464885cd824a231b851f` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:11.900781 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1daaa8d84b7464885cd824a231b851f)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:11.902738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:11.958628 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1daaa8d84b7464885cd824a231b851f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:11.959712 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1daaa8d84b7464885cd824a231b851f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:11.972184 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1daaa8d84b7464885cd824a231b851f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:11.980524 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1daaa8d84b7464885cd824a231b851f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:11.981608 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1daaa8d84b7464885cd824a231b851f'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:11.996004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1394/2845] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:48:11 2022] DELETE /identity/v3/users/f1daaa8d84b7464885cd824a231b851f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:12.000729 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:12.001208 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:12.001308 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:12.021031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:12.022586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:12.086846 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1452/2846] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:12.096743 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:12.098379 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:12.157260 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0761d-6c90-42e5-97ed-b9af81267874 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:12.161119 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:12.161985 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:12.162304 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:12.162600 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:12.173146 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:12.174597 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:12.184668 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1395/2847] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:12.475531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:12.477137 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:12.492234 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd37645-36bf-4c7c-b9a2-741fa4bf4b40 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:12.499265 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:12.501003 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:12.501433 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:12.501749 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] PATH_INFO: `/v3/projects/b05f2e96198f4abf98e10e1740149240` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:12.503412 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05f2e96198f4abf98e10e1740149240)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:12.505651 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:12.545264 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:12.566142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05f2e96198f4abf98e10e1740149240', 'request_id': 'req-17681575-07c4-4181-805d-a56ced9f70d9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:12.572620 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05f2e96198f4abf98e10e1740149240', 'request_id': 'req-17681575-07c4-4181-805d-a56ced9f70d9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:12.580969 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05f2e96198f4abf98e10e1740149240', 'request_id': 'req-17681575-07c4-4181-805d-a56ced9f70d9'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:12.593046 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] The token cache is being invalidate because project b05f2e96198f4abf98e10e1740149240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:48:12.593333 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:12.593978 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-17681575-07c4-4181-805d-a56ced9f70d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:12.595497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1453/2848] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:48:12 2022] DELETE /identity/v3/projects/b05f2e96198f4abf98e10e1740149240 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:22.180191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:22.591776 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:35.540673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:35.542384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:35.613004 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a559d7d-19bf-42f7-a45d-849ff849b6f6 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:35.666554 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:35.667176 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:35.667458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:35.667738 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:35.669750 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:35.733906 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:35.735384 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:35.745486 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1396/2849] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:41.602758 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:41.603380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:41.603380 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:41.623249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] MFA Rules not processed for user `22cfd17a6f434deca7ef77253ae3ed2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:41.624495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:41.631824 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1454/2850] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:45.740878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=68 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:51.618160 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:57.893621 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:57.895139 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:57.966124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728356b-df69-4821-9fe2-bf698dd2c7ab service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:57.970847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:57.971542 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:57.971861 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:57.972161 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] PATH_INFO: `/v3/users/39593dbdff134fb1af13b341d6c01e37` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:57.973385 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39593dbdff134fb1af13b341d6c01e37)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:57.975428 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.030123 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39593dbdff134fb1af13b341d6c01e37'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.030854 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39593dbdff134fb1af13b341d6c01e37'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.045490 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39593dbdff134fb1af13b341d6c01e37'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.052991 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39593dbdff134fb1af13b341d6c01e37'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.054116 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-014eea03-65b6-4d84-94ef-0f8838749225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39593dbdff134fb1af13b341d6c01e37'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.071349 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1397/2851] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:48:57 2022] DELETE /identity/v3/users/39593dbdff134fb1af13b341d6c01e37 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:58.258581 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.271231 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b57e9-ce50-4934-94bd-af6639257be7 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.339642 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.340242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.340551 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.340814 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] PATH_INFO: `/v3/projects/01e1704c3b1e4729820088ca9ed48368` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.342080 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e1704c3b1e4729820088ca9ed48368)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.344039 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.397287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e1704c3b1e4729820088ca9ed48368', 'request_id': 'req-be60ea6c-f1d6-471c-853f-f3f1d593c190'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.403219 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e1704c3b1e4729820088ca9ed48368', 'request_id': 'req-be60ea6c-f1d6-471c-853f-f3f1d593c190'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.412008 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e1704c3b1e4729820088ca9ed48368', 'request_id': 'req-be60ea6c-f1d6-471c-853f-f3f1d593c190'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.424206 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] The token cache is being invalidate because project 01e1704c3b1e4729820088ca9ed48368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:48:58.424494 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.425230 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-be60ea6c-f1d6-471c-853f-f3f1d593c190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:48:58.426860 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1455/2852] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:48:58 2022] DELETE /identity/v3/projects/01e1704c3b1e4729820088ca9ed48368 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:58.432694 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.433049 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.433327 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.454351 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:58.455580 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:58.526611 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1398/2853] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:58.535057 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d587-1173-41a4-93b9-92584431544b None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.535342 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d587-1173-41a4-93b9-92584431544b None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.535616 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d587-1173-41a4-93b9-92584431544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.556368 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-92e1d587-1173-41a4-93b9-92584431544b None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:58.557598 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-92e1d587-1173-41a4-93b9-92584431544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:58.565757 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1456/2854] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:48:58.570463 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.580625 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e89f0-5e57-4578-a845-e7d08988ac5a None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.585717 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.586363 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.586616 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.586885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.587561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80348) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:48:58.587933 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.589808 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.603245 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1399/2855] 10.222.0.45 () {58 vars in 1153 bytes} [Mon Aug 1 16:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:48:58.607872 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d587-1173-41a4-93b9-92584431544b None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.617842 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d587-1173-41a4-93b9-92584431544b None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.622924 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.623539 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.623845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.624109 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.624767 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.626645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.666472 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1457/2856] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:48:58 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:48:58.670779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.681878 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e241424-518e-4754-b5f3-837e5d0f13f6 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.685699 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:58.743300 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.743958 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.744219 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.744485 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.745154 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.747141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.774048 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1400/2857] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:48:58 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:48:58.777343 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.788963 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05747407-f951-430b-8bae-32ff009fe5be admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.794678 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.795402 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.795622 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.795935 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.796643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.798283 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.809495 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1458/2858] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:48:58.813837 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.825266 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4f389-a3da-4283-8bc0-dd1f503f932f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.830856 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef586a9a-9678-478f-8831-48628df14129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.831481 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef586a9a-9678-478f-8831-48628df14129 admin admin] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.831786 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef586a9a-9678-478f-8831-48628df14129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.832061 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef586a9a-9678-478f-8831-48628df14129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.832792 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef586a9a-9678-478f-8831-48628df14129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.834967 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef586a9a-9678-478f-8831-48628df14129 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.846847 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1401/2859] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:48:58.851394 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:58.862220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb8e49d-55a1-4af4-8a4f-171623cbd774 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:58.868099 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:58.868803 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.869079 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.869350 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] PATH_INFO: `/v3/projects/b6154357c0184a0cb61602f2f110b1d7/users/cc52fbd78dce4bbdada6305efb327264/roles/806f65f717d34da3aa2fba83559fdc34` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.886182 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6154357c0184a0cb61602f2f110b1d7, user_id=cc52fbd78dce4bbdada6305efb327264, role_id=806f65f717d34da3aa2fba83559fdc34)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:58.889028 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:58.904855 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1459/2860] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:48:58 2022] PUT /identity/v3/projects/b6154357c0184a0cb61602f2f110b1d7/users/cc52fbd78dce4bbdada6305efb327264/roles/806f65f717d34da3aa2fba83559fdc34 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:48:58.910666 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b20a-c9df-4e38-8eb1-d56cfb07c8df None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:58.911175 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b20a-c9df-4e38-8eb1-d56cfb07c8df None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:58.911283 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b20a-c9df-4e38-8eb1-d56cfb07c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:58.932305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-cdf4b20a-c9df-4e38-8eb1-d56cfb07c8df None None] MFA Rules not processed for user `cc7787b19b7142fdb85a77e20df6ff3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:58.933541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-cdf4b20a-c9df-4e38-8eb1-d56cfb07c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:59.005821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1402/2861] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:59.017338 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:59.019367 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:59.041335 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:59.088936 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5c424-c0a7-4562-b289-3c5a17020876 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:59.093746 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:59.094014 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:59.094262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:59.094591 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:59.106180 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:59.107632 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:59.118322 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1460/2862] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:03.634988 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:03.635514 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:03.635643 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:03.666778 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.auth.core [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] MFA Rules not processed for user `cc52fbd78dce4bbdada6305efb327264`. Rule list: `[]` (Enabled: `True`). {{(pid=80348) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:03.668082 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:03.745251 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1403/2863] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:49:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:03.749732 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:03.760852 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539dba-4e28-4680-8e11-5e5c95b7667f service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:03.774327 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:03.774897 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:03.775176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:03.775446 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:03.776176 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:03.778120 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:03.821031 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1461/2864] 10.222.0.45 () {60 vars in 1153 bytes} [Mon Aug 1 16:49:03 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:49:03.825380 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:03.838005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba0a3f-9cfa-49b2-b51b-69d1065db647 None None] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:03.856969 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:03.912947 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:03.913644 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] REQUEST_METHOD: `POST` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:03.913926 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:03.914188 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] PATH_INFO: `/v3/users` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:03.914876 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:03.917155 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:03.949846 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1404/2865] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Aug 1 16:49:03 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:49:03.958254 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:03.972880 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84fe9c-8c96-45e5-a7dc-8327467226f6 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:03.978727 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:03.979450 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:03.979818 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:03.980258 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:03.981214 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:03.984843 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:03.999590 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1462/2866] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:49:04.003783 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:04.029478 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3bf7a-fab6-448e-8f0f-b0e95fc781aa admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:04.045223 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:04.045959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:04.046212 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:04.046484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] PATH_INFO: `/v3/projects/56bdec834947448dbe20cd33f39fe2c2/users/6e6c169c8ea04f0eb72472417588f26e/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:04.063334 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bdec834947448dbe20cd33f39fe2c2, user_id=6e6c169c8ea04f0eb72472417588f26e, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:04.066085 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:04.091191 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1405/2867] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:49:04 2022] PUT /identity/v3/projects/56bdec834947448dbe20cd33f39fe2c2/users/6e6c169c8ea04f0eb72472417588f26e/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:49:04.095707 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:04.108670 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201e9e5-4045-4bcd-b057-89f500de1995 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:04.191218 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff56bfa-341b-458d-8665-3fefb187af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:04.191693 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56bfa-341b-458d-8665-3fefb187af46 admin admin] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:04.192142 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56bfa-341b-458d-8665-3fefb187af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:04.192420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56bfa-341b-458d-8665-3fefb187af46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:04.193131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff56bfa-341b-458d-8665-3fefb187af46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:04.194940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff56bfa-341b-458d-8665-3fefb187af46 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:04.207049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1463/2868] 10.222.0.45 () {58 vars in 1122 bytes} [Mon Aug 1 16:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:49:04.211576 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:04.223456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f6c6-9bb0-4f33-925c-59aa9d0eef4f admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:04.228779 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:04.229637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:04.229996 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:04.230458 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] PATH_INFO: `/v3/projects/56bdec834947448dbe20cd33f39fe2c2/users/6e6c169c8ea04f0eb72472417588f26e/roles/bdcad476ea6d4b5f9facf826c9c4b296` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:04.232732 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bdec834947448dbe20cd33f39fe2c2, user_id=6e6c169c8ea04f0eb72472417588f26e, role_id=bdcad476ea6d4b5f9facf826c9c4b296)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:04.235541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:04.248810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1406/2869] 10.222.0.45 () {60 vars in 1483 bytes} [Mon Aug 1 16:49:04 2022] PUT /identity/v3/projects/56bdec834947448dbe20cd33f39fe2c2/users/6e6c169c8ea04f0eb72472417588f26e/roles/bdcad476ea6d4b5f9facf826c9c4b296 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:49:08.722940 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deca62b-f894-4eef-a2c0-8dcb7736488e None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:08.723531 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deca62b-f894-4eef-a2c0-8dcb7736488e None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:08.723673 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deca62b-f894-4eef-a2c0-8dcb7736488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:08.763637 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-7deca62b-f894-4eef-a2c0-8dcb7736488e None None] MFA Rules not processed for user `6e6c169c8ea04f0eb72472417588f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:08.765101 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-7deca62b-f894-4eef-a2c0-8dcb7736488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:08.853441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1464/2870] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:49:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:08.868503 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:08.870444 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:09.008572 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c0950-bdee-4f68-a206-cd5e74c57075 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:09.015686 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:09.016676 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:09.016949 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:09.017217 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:09.063502 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:09.091038 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:09.092468 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:09.106501 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1407/2871] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:49:10.196926 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] REQUEST_METHOD: `POST` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:10.197423 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:10.197423 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:10.218775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.auth.core [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] MFA Rules not processed for user `cc52fbd78dce4bbdada6305efb327264`. Rule list: `[]` (Enabled: `True`). {{(pid=80349) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:10.220067 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:10.228584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1465/2872] 10.222.0.45 () {58 vars in 958 bytes} [Mon Aug 1 16:49:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:19.099167 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=10 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:20.212717 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:25.555484 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:25.557114 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:25.618760 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33ec9a-65b5-4155-9eb5-875cfaf09c72 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:25.622729 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '698e47aac57c48adb8836cec7226b039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:25.623352 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:25.623642 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:25.623940 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:25.625802 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:25.647825 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:25.650020 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:25.663500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1408/2873] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:26.800131 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:26.802187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:26.869575 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23987595-b81a-495a-bc14-27cc939a0d78 None None] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:26.874777 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:26.875429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:26.875728 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:26.876993 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:26.878445 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:26.897643 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:26.900603 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:26.915972 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1466/2874] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:35.657807 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=26 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:36.909653 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=26 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:39.106132 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:39.117003 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65beccb-15a0-4a9b-b59e-ab61841516a8 service cinder] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:39.121147 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:39.121764 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:39.122026 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:39.122301 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:39.124233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:39.134715 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:39.136305 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:39.145012 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1409/2875] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:41.451071 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:41.461587 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869ffe5-d80c-4c20-83c6-3e0263434734 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:41.466110 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:41.466737 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:41.467017 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:41.467290 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:41.477681 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:41.479123 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:41.489158 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1467/2876] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:49.140955 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:51.484976 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:50:31.072532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:31.088667 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563ada6-c750-4e02-9b63-9f7662a83087 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:31.096278 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:31.097561 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:31.097845 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:31.098113 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:31.100752 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:50:31.114558 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:31.116210 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:31.129299 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1410/2877] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:50:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:50:33.193399 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:33.208916 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448044cb-e013-48b8-a55e-e8d7d1e6c5a5 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:33.215420 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:33.216248 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:33.216560 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:33.216891 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:33.219056 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:50:33.234553 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:33.236228 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:33.248108 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1468/2878] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:50:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:50:41.123939 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=6 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:50:43.244262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:50:44.885350 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:44.895287 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6606b98-dcf4-49ec-8337-5c80543a8d11 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:44.899626 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:44.900282 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:44.900560 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:44.900868 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:44.902673 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:50:44.969216 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:44.970583 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:44.980840 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1411/2879] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:50:51.696871 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:51.698822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:50:51.717203 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d6d5b-3541-4d90-bfea-cb92fc888c16 service nova] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:51.722271 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:51.723036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:51.723313 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:51.723610 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] PATH_INFO: `/v3/users/22cfd17a6f434deca7ef77253ae3ed2f` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:51.724881 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22cfd17a6f434deca7ef77253ae3ed2f)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:51.727249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:51.781973 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22cfd17a6f434deca7ef77253ae3ed2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:51.782572 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22cfd17a6f434deca7ef77253ae3ed2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:51.791834 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22cfd17a6f434deca7ef77253ae3ed2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:51.805762 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22cfd17a6f434deca7ef77253ae3ed2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:51.812741 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22cfd17a6f434deca7ef77253ae3ed2f'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:51.818861 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1469/2880] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:50:51 2022] DELETE /identity/v3/users/22cfd17a6f434deca7ef77253ae3ed2f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:50:52.005194 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:52.015828 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dec81-11a9-4d97-bd6a-7623e87d45e4 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:52.081857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:52.082508 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:52.082787 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:52.083054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] PATH_INFO: `/v3/projects/c86fcc5e23854dfba97058deac8211d0` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:52.084313 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86fcc5e23854dfba97058deac8211d0)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:52.086734 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:52.136046 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86fcc5e23854dfba97058deac8211d0', 'request_id': 'req-0bc7ada7-faa2-4978-8084-fd7ca65426ff'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.141496 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86fcc5e23854dfba97058deac8211d0', 'request_id': 'req-0bc7ada7-faa2-4978-8084-fd7ca65426ff'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.149483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86fcc5e23854dfba97058deac8211d0', 'request_id': 'req-0bc7ada7-faa2-4978-8084-fd7ca65426ff'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.160688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] The token cache is being invalidate because project c86fcc5e23854dfba97058deac8211d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:50:52.160973 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.161646 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.163332 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1412/2881] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:50:52 2022] DELETE /identity/v3/projects/c86fcc5e23854dfba97058deac8211d0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:50:52.167517 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:52.169098 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:50:52.182893 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0965757-2dd6-4eb4-a691-2f865b063751 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:52.247459 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:52.248156 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:52.248429 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:52.248702 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] PATH_INFO: `/v3/users/78e852884c584197a68547531bcaa2b5` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:52.249944 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e852884c584197a68547531bcaa2b5)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:52.251912 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:52.304023 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e852884c584197a68547531bcaa2b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.304593 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e852884c584197a68547531bcaa2b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.314130 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e852884c584197a68547531bcaa2b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.331365 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e852884c584197a68547531bcaa2b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.333640 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:50:52.341007 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e852884c584197a68547531bcaa2b5'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.349255 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1470/2882] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:50:52 2022] DELETE /identity/v3/users/78e852884c584197a68547531bcaa2b5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:50:52.549703 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:50:52.551430 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:50:52.568614 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7ada7-faa2-4978-8084-fd7ca65426ff admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:50:52.592907 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:50:52.634365 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:50:52.635203 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:50:52.635528 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:50:52.635647 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] PATH_INFO: `/v3/projects/d5d77d23bd50495798ff519f623d1fb4` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:50:52.636895 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d77d23bd50495798ff519f623d1fb4)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:50:52.638968 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:50:52.688999 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d77d23bd50495798ff519f623d1fb4', 'request_id': 'req-ca118239-150e-462f-a149-384eaf6a9ca8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.695271 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d77d23bd50495798ff519f623d1fb4', 'request_id': 'req-ca118239-150e-462f-a149-384eaf6a9ca8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.704812 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d77d23bd50495798ff519f623d1fb4', 'request_id': 'req-ca118239-150e-462f-a149-384eaf6a9ca8'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.717705 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] The token cache is being invalidate because project d5d77d23bd50495798ff519f623d1fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80348) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:50:52.717983 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.718637 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:50:52.720233 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1413/2883] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:50:52 2022] DELETE /identity/v3/projects/d5d77d23bd50495798ff519f623d1fb4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:51:02.353987 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=4 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:02.714638 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:52.378497 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:52.380105 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:52.442441 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bec7f-0207-4643-89f8-14fb0b094d89 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:52.497391 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:52.498027 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:52.498388 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:52.498584 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:52.500473 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:52.511339 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:52.522809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:52.522809 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1471/2884] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:51:52.606943 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:52.608456 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:52.621677 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca118239-150e-462f-a149-384eaf6a9ca8 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:52.625619 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd85bf2b530ce423d8f1c010b5044dba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:52.626237 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:52.626495 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:52.626759 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:52.628570 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:52.716009 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:52.717470 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:52.729462 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1414/2885] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:51:52.801802 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:52.804513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:52.883050 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc84a7-b04c-40c5-b741-188c7b6aaa45 service neutron] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:52.888914 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:52.889548 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:52.889826 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:52.890097 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:52.892140 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:52.912242 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:52.915287 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:52.932469 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1472/2886] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:51:55.451555 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:55.453500 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:55.469708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b034b6-8fec-4740-9fb8-9cc1f7074399 service neutron] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:55.475913 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1da21c00d8494587e3ede50a5a365b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:55.476590 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:55.476888 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:55.477183 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:55.479054 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:55.492695 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:55.494294 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:55.508588 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1415/2887] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:51:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:52:02.925965 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=78 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:52:05.500723 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=40 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:52:11.032312 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:52:11.045187 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5fc1e-ad90-4703-b991-dc54ea3d82c3 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:52:11.050414 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:52:11.051036 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:11.051311 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:11.051586 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:11.053496 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:52:11.069220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:52:11.070812 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:52:11.082013 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1473/2888] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:52:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:52:21.077715 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=6 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:53:22.256959 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:22.258722 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:22.300180 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a977718-3e25-4538-b1c3-f818ab185299 service nova] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:22.309733 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:22.310389 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:22.310708 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:22.311272 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:22.316262 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:22.342592 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:22.346141 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:22.369005 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1416/2889] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:26.398345 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:26.412015 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ee24-6cbb-4f14-8b2e-63b052a4d1eb service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:26.419049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:26.419941 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] REQUEST_METHOD: `GET` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:26.420404 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:26.420850 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:26.423444 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:26.438049 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:26.439686 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:26.453349 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1474/2890] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:26.475274 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:26.477214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:26.566624 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe63bb3-420f-46d0-b305-373c5ea30f02 service placement] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:26.572102 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:26.572810 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:26.573124 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:26.573435 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] PATH_INFO: `/v3/users/cc52fbd78dce4bbdada6305efb327264` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:26.574850 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc52fbd78dce4bbdada6305efb327264)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:26.576903 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:26.637258 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc52fbd78dce4bbdada6305efb327264'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:26.637857 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc52fbd78dce4bbdada6305efb327264'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:26.665719 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc52fbd78dce4bbdada6305efb327264'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:26.679511 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc52fbd78dce4bbdada6305efb327264'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:26.680833 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc52fbd78dce4bbdada6305efb327264'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:26.702168 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1417/2891] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:53:26 2022] DELETE /identity/v3/users/cc52fbd78dce4bbdada6305efb327264 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:53:26.914215 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:26.927352 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed848-37d3-42eb-8ff1-e2aba78533e6 service placement] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:27.014698 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:27.015854 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:27.016336 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:27.016822 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] PATH_INFO: `/v3/projects/b6154357c0184a0cb61602f2f110b1d7` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:27.018895 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6154357c0184a0cb61602f2f110b1d7)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:27.022410 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:27.090132 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6154357c0184a0cb61602f2f110b1d7', 'request_id': 'req-ecae4863-c36b-4136-887b-9a11f98a0336'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.096521 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6154357c0184a0cb61602f2f110b1d7', 'request_id': 'req-ecae4863-c36b-4136-887b-9a11f98a0336'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.106775 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6154357c0184a0cb61602f2f110b1d7', 'request_id': 'req-ecae4863-c36b-4136-887b-9a11f98a0336'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.120262 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] The token cache is being invalidate because project b6154357c0184a0cb61602f2f110b1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:53:27.120578 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.121220 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.122794 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1475/2892] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:53:26 2022] DELETE /identity/v3/projects/b6154357c0184a0cb61602f2f110b1d7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:53:27.127688 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:27.129556 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:27.144415 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0acf3-f477-4134-b69a-2ea461857914 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:27.214885 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:27.215515 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:27.215791 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:27.216122 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] PATH_INFO: `/v3/users/6e6c169c8ea04f0eb72472417588f26e` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:27.217821 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6c169c8ea04f0eb72472417588f26e)` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:27.220144 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:27.229214 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:53:27.287269 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6c169c8ea04f0eb72472417588f26e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.288097 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6c169c8ea04f0eb72472417588f26e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.305089 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6c169c8ea04f0eb72472417588f26e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.314664 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6c169c8ea04f0eb72472417588f26e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.315483 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.notifications [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6c169c8ea04f0eb72472417588f26e'} {{(pid=80348) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.334525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1418/2893] 10.222.0.45 () {58 vars in 1224 bytes} [Mon Aug 1 16:53:27 2022] DELETE /identity/v3/users/6e6c169c8ea04f0eb72472417588f26e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:53:27.564784 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Authenticating user token {{(pid=80349) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:27.566743 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.fernet_utils [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80349) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:27.589527 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4863-c36b-4136-887b-9a11f98a0336 admin admin] Validating token access rules against request {{(pid=80349) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:27.693093 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7787b19b7142fdb85a77e20df6ff3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c8a8aa32f9a4584972956058f337380', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80349) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:27.693845 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:27.694347 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:27.694733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] PATH_INFO: `/v3/projects/56bdec834947448dbe20cd33f39fe2c2` {{(pid=80349) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:27.696733 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56bdec834947448dbe20cd33f39fe2c2)` {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:27.699249 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] RBAC: Authorization granted {{(pid=80349) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:27.725357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:53:27.757605 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56bdec834947448dbe20cd33f39fe2c2', 'request_id': 'req-552d3a97-ba47-4c4a-8404-93c4728f22f6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.765170 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56bdec834947448dbe20cd33f39fe2c2', 'request_id': 'req-552d3a97-ba47-4c4a-8404-93c4728f22f6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.775645 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56bdec834947448dbe20cd33f39fe2c2', 'request_id': 'req-552d3a97-ba47-4c4a-8404-93c4728f22f6'} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.796876 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] The token cache is being invalidate because project 56bdec834947448dbe20cd33f39fe2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80349) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:53:27.797357 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.798513 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG keystone.notifications [None req-552d3a97-ba47-4c4a-8404-93c4728f22f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80349) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:53:27.801136 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: [pid: 80349|app: 0|req: 1476/2894] 10.222.0.45 () {58 vars in 1233 bytes} [Mon Aug 1 16:53:27 2022] DELETE /identity/v3/projects/56bdec834947448dbe20cd33f39fe2c2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:53:37.329892 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:53:37.789899 dv-cinder-850321-2-iscsi devstack@keystone.service[80349]: DEBUG dbcounter [-] [80349] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80349) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:14.028541 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Authenticating user token {{(pid=80348) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:14.029998 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:14.197083 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba170ae-9fdc-497d-b915-95a77e400a92 admin admin] Validating token access rules against request {{(pid=80348) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:14.266397 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53482e2f81364526ae2686f42a7d2483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6264f1e9c3c649e49be50ca0b0f66888', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80348) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:14.267041 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] REQUEST_METHOD: `GET` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:14.267441 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] SCRIPT_NAME: `/identity` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:14.267824 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80348) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:14.269525 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.fernet_utils [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80348) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:14.282532 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:14.284182 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea9e33-3f38-4008-aeeb-c4b6f2afcdbd service placement] RBAC: Authorization granted {{(pid=80348) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:14.296256 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: [pid: 80348|app: 0|req: 1419/2895] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:24.291339 dv-cinder-850321-2-iscsi devstack@keystone.service[80348]: DEBUG dbcounter [-] [80348] Writing DB stats keystone:SELECT=89 {{(pid=80348) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}